Skip to content

Include df-structures descriptions in docs #993

Open

Description

@PeridexisErrant

It would be nice to include a readable version of the structures XML files in our documentation, for the benefit of script authors and others. Ideally we could parse them and translate to a nicely cross-linked format, but full support for that is more trouble than it's worth at this stage.

Here's an example implementation; I'll make a pull request once I do some more work (switch to stdlib XML library, and confirm Py2 support) and #972 is merged (so I can improve style without merge conflicts).

This demo version has a page per file, broken into sections with link targets (for use by eg. the Lua API docs). Each section is just pretty-printed and syntax-highlighted xml however, and I'm not planning to do more than that - improving the comments in the source files would be far more useful.

Activity

  1. ragundo commented on Sep 9, 2016

    @ragundo

    I've written a C++ parser for all df structures and codegen.out.xml with the purpose of showing them as a graphical viewer (like Unreal or Unity Blueprints). It generates also crossreferences between elements etc.
    The parser is complete, I'm just beginning with the graphical viewer, but it would be interesting also to generate some html files with the data. Could you provide some examples of your tool output?

  2. PeridexisErrant commented on Sep 10, 2016

    @PeridexisErrant
    ContributorAuthor

    This page has links to all the generated descriptions (jobs is a nice illustration I think).

    Really this is just pre-processing the xml a little, then pretty-printing it by sections. The pre-processing is all about normalising text (always to single-line with no leading or trailing whitespace), and moving all node text / comments / node "comment" attributes to a <comment> node (oh, and deleting <code-helper> nodes). This makes it easy to pretty-print simple by removing the <comment> tags and wrapping all lines with appropriate indentation.

    I'd love to see a better parser though, because the structures are just crying out for hyperlinks! I just stopped when I had a decent prototype, and it wasn't clear what would be most helpful. (for example, I could also delete <padding> nodes or anything where the name fits one of the (many) "unknown" patterns. And finally - in the absence of a better parser which you may have ideas for - I think it's good enough that improving the human-readability of the xml files would give better results; more and longer comments, standardising names of unknown items, and so on.

    If you're generating text output, the ideal would be to generate .rst instead of .html, so the rest of the docs can cross-reference any link targets 😄

  3. lethosor commented on Sep 10, 2016

    @lethosor
    Member

    You probably shouldn't put the trimmed filename in front of every structure - e.g. on http://peridexiserrant.neocities.org/docs-structures-test/docs/_auto/structures/d_init.html, "d_init.d_init" doesn't make sense. Structure names can only be used once across files - the separate df.*.xml files are just for organizational purposes.

  4. lethosor commented on Sep 10, 2016

    @lethosor
    Member

    I'm also a bit concerned about the size of this. The xml files are already 2 MB, and I would expect the HTML documentation to be larger than that. It could be less significant when compressed, though.

  5. PeridexisErrant commented on Sep 11, 2016

    @PeridexisErrant
    ContributorAuthor

    OK; unqualified names it is. The section link targets still have to be qualified though, or there are a few dozen name conflicts...

    For size: 6MB raw, 400KB compressed. That's about one-eighth of the current docs - nothing to sneeze at, but I think worth it as there isn't a standard way to look up the structures at the moment (advice on how to do this would be a very valuable addition to the docs).

  6. ragundo commented on Sep 11, 2016

    @ragundo

    Could you link the jobs.rst file that generates the html file?. I would like to see the source file to see if it's easy to generate it programatically.
    Thanks

  7. PeridexisErrant commented on Sep 11, 2016

    @PeridexisErrant
    ContributorAuthor

    Here it is - you can see this for anything in our docs via the "page source" link at the bottom of the HTML version.

    Also worth comparing this to the scripts pages, which use the include directive to reference text directly instead of pasting in the full content - but I'm transforming the XML so read-edit-write produces a different output.

  8. lethosor commented on Sep 11, 2016

    @lethosor
    Member

    Would it be possible to just use a single prefix for xml link targets (like "xml.d_init" instead of "d_init.d_init")?

  9. PeridexisErrant commented on Sep 13, 2016

    @PeridexisErrant
    ContributorAuthor

    No, because there are a number of duplicates between files - mostly (or possibly all?) between another file and df.globals.xml.

  10. lethosor commented on Sep 13, 2016

    @lethosor
    Member

    I don't see any struct-type or class-type in df.globals.xml. If you're referring to global objects (like d_init), maybe those could be part of a list of globals, or maybe s could be in a separate namespace.

  11. PeridexisErrant commented on Sep 13, 2016

    @PeridexisErrant
    ContributorAuthor

    Yep, so the system is literally "whatever names are in the top-level xml", so this is the globals page. Special handling is probably a good idea, but I'm not sure what it should do - presumably include links to the other pages, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions