Repository navigation
Create migration script for filesystems from LegacyResourceMapper to ResourceMapper #949
Description
Essentially, all extensionless files should get $.ttl appended.
We might want to skip files and folder trees that start with a ., such as .git/* and .htaccess. (The latter would be considered an extension, so would not be affected anyway.)
Activity
melvincarvalho commentedon Nov 21, 2018
Not necessarily a question only for @RubenVerborgh but I'd like to clarify.
Is the intention of this script to add $.ttl to files even if they are not turtle files.
As a concrete example, would
.git/HEADbecome.git/HEAD.$ttlRubenVerborgh commentedon Nov 21, 2018
Yes, as explained before, nothing observable changes for HTTP users (minus fixed bugs).
From an HTTP user's perspective, the following currently happens:
.git/HEADwill return a resource withContent-Type: text/turtle, which will then fail when parsed in the browser.That behavior, however faulty it is, will be preserved.
I realize this breaks access for file-system-based users (the exact number of which still needs to be determined); but they are free not to run that script on their folders for this very reason. (The assumption there is that file-system users are responsible for their own pod, and I cannot find exceptions to that.) We need to document this in #951.
RubenVerborgh commentedon Nov 21, 2018
For clarity: the reason of breakage is the previous design decision that extensionless files are Turtle; the only thing we are doing is interpreting data pods on disk as intended by that decision.
There is unfortunately no automated method of determining whether an extensionless file is indeed Turtle or not (given that empty files, or files with just comments, are valid Turtle too).
However, I think it would be acceptable to skip folders and files with names such as
.git,.htaccessetc; or perhaps skip anything that starts with a dot altogether. What do you think?dmitrizagidulin commentedon Nov 21, 2018
Re previous design decision about extensionless files as Turtle -- would it be another option to instead treat extensionless files as application/octet-stream, and not give them a .$ extension?
melvincarvalho commentedon Nov 21, 2018
@RubenVerborgh thank you for clarifying and the additional info.
Excuse my ignorance. Could I ask what would happen to those folders that are skipped. The files of course would remain intact. But how would the 5.0.0 server treat them -- just as before as turtle files, or as a 404, or some other mime type?
RubenVerborgh commentedon Nov 21, 2018
That breaks the HTTP interface, notably
/profile/card.RubenVerborgh commentedon Nov 21, 2018
Then they would be
application/octet-stream, as they should've been in the first place, and this gives the expected behavior for .git.Note that whatever we choose, no perfect solution exists. The question is: will people want their non-Turtle extensionless files to remain broken, or do they want them to be fixed? Arguments can be made either way.
melvincarvalho commentedon Nov 21, 2018
So, I had a quick look at the code and it seems to come from
defaultContentTypewhich is set in the ResourceMapper function. Though I suppose that in turn could be easily enough changed, or taken from the config file. That provides pretty good flexibility.Having reviewed the code, the backwards compatibility of this alot better than I had feared, so at this point many of my concerns become addressed. Mainly, that a pod owner is not locked in to one paradigm long-term.
That sounds like a good idea, if it's not too hard. Bear in mind, though, that .git is a directory and .htaccess is a file.
The main file we have issue with is /profile/card -- I wonder if in retrospect it would have been better to call it /profile/card.ttl -- tho in the back of my mind I think we may move to n3 one day ...
Thinking out loud, I wonder if it would be advantageous to check to see if a file is valid turtle before moving it.
I think you raise two valid cases, that of empty files -- what to do with them -- and the edge case of commments only. It would be tempting to leave those in place, in the knowledge that the user could delete them if they wanted.
It's possible that this is over thinking the problem. At a minimum, I'd definitely be in favour of skipping .git directories, because it's likely no good can come of that translation.
RubenVerborgh commentedon Nov 21, 2018
Indeed.
Thanks 🙂 That was the goal all along.
Okay, adding that to #949.
Told you, nothing breaks that is not already broken 🙂 Only a (currently broken) subset is affected (#950 (comment)).
I'm a big fan of extensionless files for future compatibility, but @timbl likes extensions because of the filesystem compatibility. Solid users can choose now, so that's good.
I don't think it helps because empty files and files with only
#.*lines are also valid Turtle files.+1
Add migration tool for migrating legacy resources