You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be nice to include a readable version of the structures XML files in our documentation, for the benefit of script authors and others. Ideally we could parse them and translate to a nicely cross-linked format, but full support for that is more trouble than it's worth at this stage.
Here's an example implementation; I'll make a pull request once I do some more work (switch to stdlib XML library, and confirm Py2 support) and #972 is merged (so I can improve style without merge conflicts).
This demo version has a page per file, broken into sections with link targets (for use by eg. the Lua API docs). Each section is just pretty-printed and syntax-highlighted xml however, and I'm not planning to do more than that - improving the comments in the source files would be far more useful.
I've written a C++ parser for all df structures and codegen.out.xml with the purpose of showing them as a graphical viewer (like Unreal or Unity Blueprints). It generates also crossreferences between elements etc.
The parser is complete, I'm just beginning with the graphical viewer, but it would be interesting also to generate some html files with the data. Could you provide some examples of your tool output?
This page has links to all the generated descriptions (jobs is a nice illustration I think).
Really this is just pre-processing the xml a little, then pretty-printing it by sections. The pre-processing is all about normalising text (always to single-line with no leading or trailing whitespace), and moving all node text / comments / node "comment" attributes to a <comment> node (oh, and deleting <code-helper> nodes). This makes it easy to pretty-print simple by removing the <comment> tags and wrapping all lines with appropriate indentation.
I'd love to see a better parser though, because the structures are just crying out for hyperlinks! I just stopped when I had a decent prototype, and it wasn't clear what would be most helpful. (for example, I could also delete <padding> nodes or anything where the name fits one of the (many) "unknown" patterns. And finally - in the absence of a better parser which you may have ideas for - I think it's good enough that improving the human-readability of the xml files would give better results; more and longer comments, standardising names of unknown items, and so on.
If you're generating text output, the ideal would be to generate .rst instead of .html, so the rest of the docs can cross-reference any link targets 😄
I'm also a bit concerned about the size of this. The xml files are already 2 MB, and I would expect the HTML documentation to be larger than that. It could be less significant when compressed, though.
OK; unqualified names it is. The section link targets still have to be qualified though, or there are a few dozen name conflicts...
For size: 6MB raw, 400KB compressed. That's about one-eighth of the current docs - nothing to sneeze at, but I think worth it as there isn't a standard way to look up the structures at the moment (advice on how to do this would be a very valuable addition to the docs).
Could you link the jobs.rst file that generates the html file?. I would like to see the source file to see if it's easy to generate it programatically.
Thanks
Here it is - you can see this for anything in our docs via the "page source" link at the bottom of the HTML version.
Also worth comparing this to the scripts pages, which use the include directive to reference text directly instead of pasting in the full content - but I'm transforming the XML so read-edit-write produces a different output.
I don't see any struct-type or class-type in df.globals.xml. If you're referring to global objects (like d_init), maybe those could be part of a list of globals, or maybe s could be in a separate namespace.
Yep, so the system is literally "whatever names are in the top-level xml", so this is the globals page. Special handling is probably a good idea, but I'm not sure what it should do - presumably include links to the other pages, though.
Activity
ragundo commentedon Sep 9, 2016
I've written a C++ parser for all df structures and codegen.out.xml with the purpose of showing them as a graphical viewer (like Unreal or Unity Blueprints). It generates also crossreferences between elements etc.
The parser is complete, I'm just beginning with the graphical viewer, but it would be interesting also to generate some html files with the data. Could you provide some examples of your tool output?
PeridexisErrant commentedon Sep 10, 2016
This page has links to all the generated descriptions (
jobsis a nice illustration I think).Really this is just pre-processing the xml a little, then pretty-printing it by sections. The pre-processing is all about normalising text (always to single-line with no leading or trailing whitespace), and moving all node text / comments / node "comment" attributes to a
<comment>node (oh, and deleting<code-helper>nodes). This makes it easy to pretty-print simple by removing the<comment>tags and wrapping all lines with appropriate indentation.I'd love to see a better parser though, because the structures are just crying out for hyperlinks! I just stopped when I had a decent prototype, and it wasn't clear what would be most helpful. (for example, I could also delete
<padding>nodes or anything where the name fits one of the (many) "unknown" patterns. And finally - in the absence of a better parser which you may have ideas for - I think it's good enough that improving the human-readability of the xml files would give better results; more and longer comments, standardising names of unknown items, and so on.If you're generating text output, the ideal would be to generate
.rstinstead of.html, so the rest of the docs can cross-reference any link targets 😄lethosor commentedon Sep 10, 2016
You probably shouldn't put the trimmed filename in front of every structure - e.g. on http://peridexiserrant.neocities.org/docs-structures-test/docs/_auto/structures/d_init.html, "d_init.d_init" doesn't make sense. Structure names can only be used once across files - the separate
df.*.xmlfiles are just for organizational purposes.lethosor commentedon Sep 10, 2016
I'm also a bit concerned about the size of this. The xml files are already 2 MB, and I would expect the HTML documentation to be larger than that. It could be less significant when compressed, though.
PeridexisErrant commentedon Sep 11, 2016
OK; unqualified names it is. The section link targets still have to be qualified though, or there are a few dozen name conflicts...
For size: 6MB raw, 400KB compressed. That's about one-eighth of the current docs - nothing to sneeze at, but I think worth it as there isn't a standard way to look up the structures at the moment (advice on how to do this would be a very valuable addition to the docs).
ragundo commentedon Sep 11, 2016
Could you link the jobs.rst file that generates the html file?. I would like to see the source file to see if it's easy to generate it programatically.
Thanks
PeridexisErrant commentedon Sep 11, 2016
Here it is - you can see this for anything in our docs via the "page source" link at the bottom of the HTML version.
Also worth comparing this to the scripts pages, which use the
includedirective to reference text directly instead of pasting in the full content - but I'm transforming the XML so read-edit-write produces a different output.lethosor commentedon Sep 11, 2016
Would it be possible to just use a single prefix for xml link targets (like "xml.d_init" instead of "d_init.d_init")?
PeridexisErrant commentedon Sep 13, 2016
No, because there are a number of duplicates between files - mostly (or possibly all?) between another file and
df.globals.xml.lethosor commentedon Sep 13, 2016
I don't see any struct-type or class-type in df.globals.xml. If you're referring to global objects (like d_init), maybe those could be part of a list of globals, or maybe s could be in a separate namespace.
PeridexisErrant commentedon Sep 13, 2016
Yep, so the system is literally "whatever names are in the top-level xml", so this is the globals page. Special handling is probably a good idea, but I'm not sure what it should do - presumably include links to the other pages, though.