Repository navigation
Persist Content-Type of resources without file extensions #275
Description
Activity
Do we have a spec for how resources live on the filesystem? It seems like something a lot of tools might want to agree on, even though it's not the wire protocol, and not everyone has to use a filesystem, of course. My copy-web-to/from-filesystem tool needs the same thing.
@sandhawke - What about solid servers that don't use the filesystem at all, like Meccano?
There shouldn't really be a spec for that, since the only way apps (like the migrator app) should be accessing those files is through the API.@dmitrizagidulin I'm just talking about the systems that happen to want to join in, in using the same filesystem conventions. Meccano may want to use it, too, for its export-in-a-zip-file and import-from-a-zip-file functions.
Other elements of this spec would be where metadata and acls go. I'm thinking ...$describedby.ttl and ...$acl.ttl (and this can be extended, where basically the word after the $ is the rel text)
So if I GET /foo and get html, and there's a rel=describedby to /foo,meta.ttl and a rel=acl to /foo,acl then I'll create three files
- foo$.html for the main data
- foo$describedby.ttl for the metadata (note that its name is NOT what the resource name was)
- foo$acl.ttl for the acl (again, this filename bears no relation to where the ACL was served from)
How's that sound, @timbl ?
I agree with @dmitrizagidulin. Although it is nice to have everyone being on the same page on how to represent data on the file system, this is beyond the solid spec - in other words, everyone can implement it the way they want, the import/export that you mention should be done in a ldp way, exactly to avoid this problem, where all of this is abstracted.
I completely agree it wouldn't be part of solid, per se. Certainly it wouldn't affect normal clients at all. That's why I asked if we "have a spec". (Although I guess if you view solid as a collection of specs my phrasing is ambiguous.)
The "filename conventions" or whatever spec would be both useful and potentially confusing. As you guys suggest, people might think it's part of solid and somehow think they have to follow it. I would think the solid spec could be clear enough about that. Solid could point to it and explain what it's for. I'm sure you'd agree it would actually be useful if gold and ldnode and solid-copy (my current name for the command line copy client) all happily used the same filesystem conventions. At the same time, of course, when folks aren't sharing a filesystem between multiple people or multiple tools, they can do as they like, and even if they are they can use different filename conventions and still be solid.
One point @sandhawke,
solid-copybecause it is an "LDP" copy, should not know about how the server store filesIndeed, solid-copy uses solid to talk to the remote server. But it needs some convention for how it is going to store files on the local filesystem. And if that convention is the same as gold and ldnode, then it'll avoid some pain for people running their own servers.
IMO all the RDF Sources should allow content negotiation. How they get stored in filesystem, document database, quads store etc. should not concern clients and stay a choice of internal plumbing used by particular server.
I would recommend that ldnode simply stores all the RDF documents as Normalized RDF.
In my prototype of a server I pass documents through https://github.com/rdf-ext/rdf-normalize before storing them. Later I parse them again with https://github.com/rdf-ext/rdf-parser-n3 and depending on content negotiation response gets serialize with one of https://github.com/rdf-ext/rdf-ext#serializersProblem: When writing a resource without a file extension, there is currently no way for the server to remember what its content-type was. So, when it's next requested, the server doesn't know what that resource's content type is, and returns text/turtle by default.
For RDF Source, shouldn't the server just store the named graph and don't give any importance to the content type of the payload which provided the g-text ?
Only Non-RDF Source seems to need a way to save information about their content type. Maybe RDF Source which they have desribedby relation with, could include triple with predicate similar to as:mediaType
For RDF Source, shouldn't the server just store the named graph
How do you picture the server storing the named graph? For node-solid-server, it stores them as files. And files need extensions, otherwise how is the server supposed to know how to serve them back? (The server does not look inside the file, incidentally).
And yes, this is mainly a problem for non-RDF sources.
how is the server supposed to know how to serve them back?
For RDF Source server could have configured default media type e.g text/turtle. I assume that we talk about cases where client doesn't include Accept header to content negotiate preferred representation.
And yes, this is mainly a problem for non-RDF sources.
I think RDF Source and Non-RDF Source deserve different treatment
Only Non-RDF Source seems to need a way to save information about their content type. Maybe RDF Source which they have desribedby relation with, could include triple with predicate similar to as:mediaType
I just discover this issue and I this is the way I chose to store the content type of resources posted or put by clients: the contents of the
content-typeheader of the POST or PUT request is added to the.metaassociated to the new/modified resource. I usedc:formatthough, but without strong conviction about this property rather than another one.15 remaining items
- added 2 commits that reference this issue
on Sep 12, 2017 Alright, sounds reasonable.
Aside: re HTML+RDFa documents (or dokieli-like thing if you will), we'll see how that works out. Its articles, annotations, notifications do use relative values for some of its subjects/objects. Normally, the base URL will resolve to the URL of the document. So, I think it is possible that .html for instance will leak out. That can be overridden of course when processed, but tools like dokieli can't arbitrarily - and in most cases shouldn't - make the call on what is the intended URL and what's preferable. Needless to say, this is not to suggest that seeing or using
.htmlis an error in any way.I'd like to know what happens with GET
/foo(withfoo.htmlon disk). Does it 301/302, or 200 with optionallyContent-Location? If I understand correctly, GET/foois a 404.. and the POST from earlier probably returnedContent-Location: /foo.html.. so, only GET/foo.htmlis a 200?So, I think it is possible that .html for instance will leak out.
Yes, it will currently leak out. But that's not a problem if we treat URLs as opaque identifiers. I.e., the server used to return
foo12345and we did not break any of that apart, now the server will returnfoo12345.htmland we still don't break it apart. To the client, the.htmlshould not have any meaning, it might as well have been `foo12345-xyz' or anything else.but tools like dokieli can't arbitrarily - and in most cases shouldn't - make the call on what is the intended URL and what's preferable.
Fully agree.
I'd like to know what happens with GET /foo (with foo.html on disk). Does it 301/302, or 200 with optionally Content-Location?
Nothing at the moment, because we still need a good mapping between URIs and the file system (#551).
(Creating a tracking issue for PRs #159 and #229)
Problem: When writing a resource without a file extension, there is currently no way for the server to remember what its content-type was. So, when it's next requested, the server doesn't know what that resource's content type is, and returns
text/turtleby default.Potential Solutions:
.metafile. Have the server check the.meta(if the resource doesn't have a file extension), discover the content type, and return it with the response.articlewith an HTML content type, the file will be namedarticle.$.htmlon disk. When reading the resource, ldnode detects the special extension (and filters it out), and returns the resource with the correct content-type.