Repository navigation
Qt Help: hand the viewers the site stylesheet and a dark scheme - #52
Open
cristianadam wants to merge 2 commits into
Open
cristianadam wants to merge 2 commits into
cristianadam wants to merge 2 commits into
Conversation
The Qt Help output is read in Qt Assistant and in Qt Creator, and the pass that built it was written when Assistant rendered with QTextBrowser, which follows no stylesheet and knows a small corner of CSS. So the pass flattened cppreference's stylesheet into `style` attributes, threw the classes away and rewrote the layout as nested tables, and the pages read like it. Both viewers render with litehtml now -- Assistant since Qt 6.2, Qt Creator since 4.11 -- which does follow a stylesheet, so give them the site's own and let them lay the pages out the way a browser does. The stylesheet. The pages keep their markup and their classes and link one file, which `commands/qch_css.py` builds out of `ext.css`, `site_modules.css`, the seven distinct `<style>` blocks that turn out to lie behind six thousand pages, and `qch-viewer.css` for what a help pane needs on top. The archive drops from forty-four megabytes to thirty-two on the way, most of it the navigation menus every page carried twice: litehtml has `:hover` and would have worked them, but a help pane is a poor place to reach for a menu and both viewers put a Contents and an Index beside the page already. A color scheme. Asked for `offline-simple.css`, both viewers serve `offline-dark.css` in its place when their own theme is dark, and `offline.css` when it is light; Qt Creator 21 does the same for a page that asks for `offline.css` outright. So the pages ask for the first and the light sheet is written under both of the others. The dark sheet is the light one followed by a counterpart for every color in it, mapped from the light color rather than written out by hand, so that it keeps up with cppreference on its own; the mapping holds a dark surface to WCAG AA against the body text and keeps chroma rather than HSL saturation, so that a pale tint does not come out vivid. The counterpart is written for the declaration the light cascade settles on and not for every declaration along the way: the derived rules go at the end, where they outrank the whole of the sheet above them, so a color that something further down takes back would be a color the dark page has and the light page has not -- `ext.css` gives inline `code` a background and `site_modules.css` makes it transparent again, and deriving from the first of those would put a grey box behind every type name in the archive. That is read across the sheets together, because the cascade runs across them too. Rules the site wrote for `print` are left to it. What no rule can reach is the color a wiki template wrote into a `style` attribute -- the green and red of the compiler-support tables -- so `preprocess_qch_html` names each of those with a class and the dark rules for them say `!important`, which litehtml honours over the attribute as the cascade asks. Type sizes. The site is written against a browser's 16px default and takes a fifth off its content to read at about 13px; a help pane starts from the font the reader chose for it, which is already the size they want to read at, so the fifth was coming off twice and the page came out small. Let the content sit at the reader's size, and pull the proportions around it towards the middle by raising each to a power. The sizes of a cascade multiply and a root distributes over a product, so compressing every declared proportion by the same exponent compresses the size each element ends up at by exactly that exponent, and nothing has to work out what an element inherits. It does have to be every declared proportion, the eleven thousand a wiki template wrote into a `style` attribute along with the ones in the sheets, or whatever hangs under one of those is a sixth smaller than the page around it. The content lands at 0.89 of the reader's size and a page title at 1.26 of it -- 14px and 20px of a 16px Help font -- which is the scale Qt's own documentation reads at, and the whole of it still follows the Help font size. Fonts. A font stack is worth its first name and no more: litehtml hands the whole list to the viewer, the viewer hands it to `QFont::setFamilies`, and that list is for glyph coverage rather than for choosing a face -- Qt resolves the first family and substitutes its own default when it is missing instead of trying the next. cppreference names DejaVu first throughout, which hardly anyone outside Linux has, so every code sample came out in a proportional font. Each stack is cut down to the first generic in it, those being the names qlitehtml answers with faces of its own; a stack that comes down to `sans-serif` asks to inherit instead, which is the reader's font at the root and still takes an element back out of a monospace rule further in. The `style` attributes are cut down with them: three thousand of those name Georgia first, which is no better installed than DejaVu. Courier, which is what a viewer gives us for `monospace`, draws larger than the proportional face beside it, so a type name in the middle of a sentence stands up out of the line; ask for 85% of it. That belongs to the face and not to the text, so it is taken where the monospace starts and not again inside it -- and monospace nests on every page, a `<pre>` inside a highlighted block inside a monospace cell being three deep. There is no way to ask CSS for "unless you are already inside one", so the pairs are written out. An element that is set proportionally goes back to the size it inherits for the same reason: the member names of the declaration tables carry a monospace rule and a proportional one on the same element, and the face they end up in is the reader's own, which needs no correcting. What a stylesheet cannot do is left to `preprocess_qch_html`. A MediaWiki template asks a cell to span 142 rows as a way of saying "all of them", and litehtml lays a table out as a flat run with no notion of the row groups a browser stops a `rowspan` at, so the markers that number the overloads ended up nowhere near them; say what we mean. litehtml has no `writing-mode` either, so the column headings of the compiler-support tables came out lying across one another: the stylesheet lays them out across and lets them wrap, and the row they were standing in no longer needs the height, which no selector can reach from the heading. Images. Only the HTML was carried into the archive, so every figure the pages draw on -- the diagrams of what `begin` and `end` point at, the inheritance charts of the exception types -- arrived broken. The ones the pages and the stylesheet ask for are taken now, the page icon among them, which is fewer than the site has. The skin images came down with their cache-busting query percent-encoded into the file name, and nothing renamed the files to match, so those references are mended on the way in. A raster image is handed the white background it was drawn against, without which its lettering disappears. A drawing does better than being plated, because the viewer tells it which page it is on: both Qt Assistant and Qt Creator put `class="dark"` on the root <svg> element when their own theme is dark. So `commands/qch_svg.py` gives each one something to answer with. Name every color it is drawn with by a class, state under that class the color it already had, and state the dark counterpart under `svg.dark`, mapped by the same palette the stylesheet is mapped by -- a dark color lightened the way the page's text is, a light one darkened the way a surface is, which carries a drawing over whole. The marks lighten, the boxes behind them darken, and the grey of a muted label stays the step off the lettering that it was. Nothing is taken away. A color stated as a presentation attribute is left where it stands, since a rule already outranks one and it is what a renderer that reads no CSS at all goes on drawing; only a color in a `style` attribute has to be moved, that outranking every rule. A viewer that does not know the trick never matches `svg.dark` and draws exactly what it drew before -- which is why the drawings are also handed a backdrop. They have no ground of their own; they were borrowing the page's, and the plate is now kept off them. Three things Qt's SVG renderer settles rather than the standard. It honours a class selector but not an attribute one, so the classes are written out rather than the colors matched where they stand. It applies no rule to the root <svg> element at all, so what the root is drawn in is restated on a <g> put around everything it holds -- and the root must not be given a class in any case, the viewer writing one onto it and a second `class` attribute beside the first not being XML. And it reads a gradient stop's color off the stop and never asks the stylesheet, so a gradient cannot be restyled; `fill: url(#...)` is honoured, so each one is copied into a dark twin and the reference is switched instead. The lettering asks for no color at all and takes the black SVG paints with when nothing says otherwise, so that is said outright where a mark would otherwise inherit it -- but not inside <defs> and the like, where a mark is stood up by a <use> somewhere else and takes its color from there, which is how the plot markers of the graphs are colored. The same pass gives a drawing the pixels to be drawn sharply. Qt rasterises an SVG at the size the file declares and takes no account of the ratio of the display, so on a Retina screen the bitmap is blown up to twice the pixels it was drawn with and a hairline diagram comes out soft; the viewer's own zoom multiplies on top of that. So declare the drawing at twice the size and leave what it is drawn at to the page, which sizes every one of these figures with a `width` and a `height` on the <img> element -- litehtml maps the two onto the CSS properties and only falls back on the file's own size when neither is given. Nothing moves; there is simply more of the drawing to draw, and a display that is not Retina scales it back down smoothly. Where a drawing states no `viewBox` the box it has been drawn in all along is stated, without which a larger `width` would only lay out a larger canvas. Against the sixty-four drawings the archive carries, Qt's renderer gives back the light rendering pixel for pixel, every one of them goes dark, and each is rasterised at twice the pixels the page lays it out in.
Builds the .qch on every push and attaches it to the release when a version is tagged, so the archive people actually download is built the same way every time rather than on whichever machine happened to have a qhelpgenerator on it. The raw copy of the website is the awkward part: it is not in the repository, it is 475 MB, and crawling en.cppreference.com on every run is what the README asks us not to do. It does not have to be crawled, though -- `make release` already packs `reference/` into `cppreference-doc-<version>.tar.xz`, so the workflow takes the site copy out of an earlier release's source archive, which is 13 MB and unpacks in about a minute. Refreshing the content for a real release is then a matter of running `make source` locally and publishing a new source archive; the workflow picks it up as the latest on its own. Which repository that archive comes from cannot just be this one. A fork has no releases until it makes one, so the run asks GitHub what this repository was forked from and works its way up until it finds a source archive, which keeps the workflow building in a fork while still preferring a release of its own once there is one. The dispatch inputs name a repository and a tag outright when some other content is wanted. An artifact and a release asset want different things. GitHub zips an artifact whatever it is handed and serves the result under the artifact's name, so the artifact is the .qch itself -- handing it the release archives makes a zip holding an archive holding the one file anybody wanted, compressed twice over. A release asset is served as it is, so the tar.xz and the zip `make release` writes are built only for a tag, where they are the real thing. qhelpgenerator comes from Ubuntu's `qt6-documentation-tools`, whose path is asked of dpkg rather than written out, Qt keeping the binary under /usr/lib/qt6/libexec and being free to move it. A tag that disagrees with VERSION in the Makefile stops the run, rather than publishing archives whose names do not match the release they hang under. The .qch is a SQLite database, so once built it is asked whether the pages, the images and both stylesheets arrived in it. pylint is left out of the checks, unlike `test.sh`: `config/pylintrc` predates pylint 3 and is rejected across the whole repository, which is its own change to make.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Qt Help pass was written when Qt Assistant rendered with
QTextBrowser,which follows no stylesheet and understands a small corner of CSS. So it
flattened cppreference's stylesheet into
styleattributes, threw the classesaway and rewrote the layout as nested tables — and the pages read like it.
Both viewers that read the archive render with litehtml now (Assistant since
Qt 6.2, Qt Creator since 4.11), and litehtml does follow a stylesheet. This
hands them the site's own and lets them lay the pages out much as a browser
does, adds a dark scheme that is derived from the light colors rather than
written by hand, and fixes the images, which were not being copied into the
archive at all.
The archive gets smaller on the way: 44 MB → 32 MB, most of it the
navigation menus every page was carrying twice over.
What's in it
commands/preprocess_cssless.pyis replaced by three focused modules:commands/qch_css.pycommands/preprocess_qch_html.pycommands/qch_svg.pyThe stylesheet. Pages keep their markup and classes and link one file,
built from
ext.css,site_modules.css, the seven distinct<style>blocksthat turn out to lie behind six thousand pages, and
qch-viewer.cssfor what ahelp pane needs on top.
A dark scheme. Asked for
offline-simple.css, both viewers serveoffline-dark.csswhen their own theme is dark andoffline.csswhen it islight, so the pages ask for the first and the light sheet is written under both
of the others. The dark sheet is the light one followed by a counterpart for
every color in it, mapped from the light color rather than written out — so it
keeps up with cppreference's stylesheet on its own. The mapping holds a dark
surface to WCAG AA against the body text and preserves chroma rather than HSL
saturation, so a pale tint does not come out vivid.
A counterpart is written only for the declaration the light cascade settles on,
read across the sheets together. Deriving from every declaration along the way
would put colors on the dark page that the light page does not have —
ext.cssgives inline
codea background andsite_modules.cssmakes it transparentagain, and following the first would put a grey box behind every type name in
the archive. Rules the site wrote for
printare left alone.Type sizes and fonts. The site is written against a browser's 16px default
and takes a fifth off its content; a help pane starts from the font the reader
chose, which is already the size they want, so the fifth was coming off twice.
Every declared proportion is now compressed by raising it to a power — sizes
multiply down a cascade and a root distributes over a product, so compressing
each declared step compresses what every element ends up at by the same
exponent, without anyone working out what each element inherits. Content lands
at 0.89 of the reader's size and a page title at 1.26, which is the scale Qt's
own documentation reads at, and the whole of it still follows the Help font
size.
A font stack is worth its first name and no more: litehtml hands the list to
the viewer, which hands it to
QFont::setFamilies, and that list is for glyphcoverage rather than for choosing a face — Qt resolves the first family and
substitutes its default when it is missing instead of trying the next.
cppreference names DejaVu first throughout, so every code sample was coming out
in a proportional font. Each stack is cut down to the first generic in it.
Images. Only the HTML was carried into the archive, so every figure arrived
broken. The ones the pages and the stylesheet ask for are now copied (100 of
them), the page icon among them. The skin images came down with their
cache-busting query percent-encoded into the file name, and those references
are mended on the way in.
Diagrams. A help viewer puts
class="dark"on the root<svg>when itstheme is dark, so each drawing can answer for itself: every color it is drawn
with is named by a class, the color it had is stated under that class, and the
dark counterpart under
svg.dark. A viewer that does not know the trick nevermatches
svg.darkand draws exactly what it drew before. Gradients cannot berestyled — Qt's renderer reads a stop's color off the stop — so each is copied
into a dark twin and the
fill: url(#...)reference is switched instead. Thesame pass declares each drawing at twice its size so it rasterises sharply on a
Retina display, which moves nothing because every figure is sized by
widthand
heighton the<img>.What a stylesheet cannot do is left to
preprocess_qch_html.py: aMediaWiki template asks a cell to span 142 rows to mean "all of them", and
litehtml lays a table out as a flat run with no row groups to stop at, so the
overload markers ended up nowhere near the declarations they number. Same for
writing-mode, which litehtml lacks, so the compiler-support headings werelying across one another.
CI
The second commit adds
.github/workflows/qch.yml, which builds the.qchonevery push and attaches it to the release when a version is tagged.
The raw site copy is the awkward part — it is not in the repository, it is
475 MB, and crawling en.cppreference.com on every run is what the README asks
us not to do. It does not have to be crawled:
make releasealready packsreference/intocppreference-doc-<version>.tar.xz, so the workflow takesthe site copy out of an earlier release's source archive (13 MB, about a
minute). Refreshing the content for a real release is then a matter of running
make sourcelocally and publishing a new source archive, which the workflowpicks up as the latest on its own. It also walks up the fork chain, so it
builds in a fork that has no releases of its own yet.
This commit is independent of the first and can be dropped or split out if
you'd rather take the rendering change on its own.
New dependency
cssutils(added torequirements.txtand the README's dependency list;Debian/Ubuntu ship it as
python3-cssutils).Worth flagging:
cssutils==2.15.0requires Python ≥ 3.8, while.travis.ymlstill lists 3.6 and 3.7. That config looks stale — it is pinnedto
branches: only master— but it would fail atpip install -r requirements.txtif it is still running anywhere. Happy to bump it in this PRif you want.
Testing
python -m unittest discover— 173 tests, all passing (thepreprocess_csslesssuite is replaced bytest_qch_css.py,test_qch_svg.pyandtest_preprocess_qch_html.py)flake8 --config=config/flake8rccleanmake doc_qchover the whole site: 6638 pages, 100 images, 64 SVGs,32 MB archive, 6742 files in it. Opened in Qt Creator in both themes.
light rendering pixel for pixel, every one goes dark, and each rasterises at
twice the pixels the page lays it out in.
pylintis not run in the new workflow:config/pylintrcpredates pylint 3and is rejected across the whole repository, which is its own change to make.
Screenshots