Repository navigation
Chinese translations for HTML take over an hour each #209
Description
Filtering the recent HTML rebuilds to Chinese only, we see that the average build takes 1h 6m 43s, with the shortest at 54m 25s and longest at 1h 29m 58s. This is far out of line of every other language, which take under five minutes on average. For each Python version, one Chinese build takes longer than all other languages put together.
@mattwang44 has investigated this somewhat in python/python-docs-zh-tw#167, though without the pathalogical times we're observing in docsbuild.
We should identify what is responsible for this, and fix it.
A
Full list of times for HTML-only builds for zh-CN or zh-TW:
| Start | Version | Language | Build | Trigger |
|---|---|---|---|---|
| 2024-09-30 18:06 UTC | 3.14 | zh-cn | 1h 0m 32s | new translations |
| 2024-09-30 19:06 UTC | 3.14 | zh-tw | 1h 15m 51s | new translations |
| 2024-09-30 21:05 UTC | 3.13 | zh-cn | 56m 49s | new translations |
| 2024-09-30 22:02 UTC | 3.13 | zh-tw | 1h 10m 57s | new translations |
| 2024-09-30 23:54 UTC | 3.12 | zh-cn | 55m 9s | Doc/ has changed |
| 2024-10-01 00:49 UTC | 3.12 | zh-tw | 1h 9m 21s | Doc/ has changed |
| 2024-10-01 03:13 UTC | 3.13 | zh-cn | 54m 51s | Doc/ has changed |
| 2024-10-01 05:16 UTC | 3.14 | zh-cn | 1h 4m 6s | new translations |
| 2024-10-01 06:27 UTC | 3.13 | zh-cn | 1h 11m 29s | new translations |
| 2024-10-01 09:02 UTC | 3.14 | zh-cn | 56m 53s | new translations |
| 2024-10-01 09:59 UTC | 3.14 | zh-tw | 1h 15m 57s | Doc/ has changed |
| 2024-10-01 11:15 UTC | 3.13 | zh-cn | 58m 51s | new translations |
| 2024-10-01 12:57 UTC | 3.12 | zh-cn | 54m 25s | Doc/ has changed |
| 2024-10-01 13:51 UTC | 3.12 | zh-tw | 1h 15m 16s | Doc/ has changed |
| 2024-10-01 16:04 UTC | 3.14 | zh-cn | 58m 17s | new translations |
| 2024-10-01 17:02 UTC | 3.14 | zh-tw | 1h 19m 26s | Doc/ has changed |
| 2024-10-01 18:21 UTC | 3.13 | zh-cn | 59m 8s | new translations |
| 2024-10-01 19:21 UTC | 3.12 | zh-tw | 1h 14m 37s | Doc/ has changed |
| 2024-10-01 22:04 UTC | 3.14 | zh-cn | 1h 2m 8s | new translations |
| 2024-10-01 23:06 UTC | 3.14 | zh-tw | 1h 12m 55s | Doc/ has changed |
| 2024-10-02 05:56 UTC | 3.12 | zh-cn | 1h 0m 33s | Doc/ has changed |
| 2024-10-02 06:57 UTC | 3.12 | zh-tw | 1h 22m 37s | Doc/ has changed |
| 2024-10-02 09:21 UTC | 3.14 | zh-cn | 1h 7m 2s | new translations |
| 2024-10-02 10:38 UTC | 3.13 | zh-cn | 1h 2m 27s | new translations |
| 2024-10-02 16:16 UTC | 3.14 | zh-cn | 1h 7m 58s | new translations |
| 2024-10-02 17:24 UTC | 3.14 | zh-tw | 1h 29m 58s | Doc/ has changed |
| 2024-10-02 18:54 UTC | 3.13 | zh-cn | 1h 3m 49s | new translations |
Activity
AA-Turner commentedon Oct 3, 2024
The following is a heavily trimmed log for today's
zh-TWbuild:The full
reading sourcesphase takes less than two minutes, roughly as expected.The writing phase starts at 09:26:11 and ends at 10:47:11, taking 1h21m.
Looking at the duration of each chunk in detail:
All of these are taking too long, but the "library/wsgiref .. tutorial/interactive" chunk is the clear outlier at 40 minutes.
log for zh-TW
log for en (baseline)
log for es (reference translation)
rffontenelle commentedon Oct 3, 2024
Not much a contribution, but just to mention that these build times are far from e.g. Sphinx's own translated docs build. See sphinx-zh-cn (6 minutes or less) and sphinx-zh-tw (3 minutes or less)
rffontenelle commentedon Oct 3, 2024
zh_CN team seems to be caching the build to reduce time to ~2 min in their GitHub Actions runs (example). They use actions/cache/restore, build docs, and then actions/cache/save.
AA-Turner commentedon Oct 3, 2024
We need to rebuild from scratch (
-Ea) each time indocsbuild, so we can't use caching.A
jayaddison commentedon Oct 8, 2024
@AA-Turner is it possible to produce a list of the timestamps on the written files to narrow in a bit further than
library/wsgiref .. tutorial/interactiveto identify elapsed-time outliers?AA-Turner commentedon Oct 9, 2024
From the latest run:
Recursive
statof every file in the output folder (I've removed_sourcesetc):hugovk commentedon Oct 9, 2024
Highlighting some of the larger time gaps. One is a huge file, many are small, so perhaps these time gaps are misleading.
~22 minutes for
interactive.htmlhttps://docs.python.org/zh-cn/3.14/tutorial/interactive.html:~18 minutes for
changelog.html. Perhaps not surprising as this is a huge file. The html is 5.2M in English builds. It's also untranslated except for the header https://docs.python.org/zh-cn/3.14/whatsnew/changelog.html~9 minutes https://docs.python.org/zh-cn/3.14/library/winsound.html:
~8 minutes https://docs.python.org/zh-cn/3.14/library/textwrap.html:
~7 minutes https://docs.python.org/zh-cn/3.14/library/readline.html:
~5 minutes https://docs.python.org/zh-cn/3.14/library/math.html:
~5 minutes https://docs.python.org/zh-cn/3.14/library/grp.html:
mattwang44 commentedon Oct 9, 2024
if combining the logs and the stat metrics, you'll get
I wonder what does the timestamp in
1728441249 :2024-10-09 02:34:09.701973645 +0000 ./tutorial/interactive.htmlstands for? is it HTML file creation time or last modified time?If the timestamp stands for file creation time (and I assume sphinx-build appends the content to the file after the file is created and induces the gap), those timestamps would then make sense to me.
AA-Turner commentedon Oct 10, 2024
I've had some time today to do some profiling using py-spy:
$ py-spy record --format speedscope -o speedscope.json -- python -m sphinx build -M html . build -aET -D locale_dirs=locale-zh-tw -D language=zh_TW -D gettext_compact=0This reveals that the vast majority (93%) of the build duration was spent in
SearchChinese.stem. After playing around withPyStemmer, it turns out there's a much more mundane explanation.SearchChinese.latin_termsis a collection of latin terms that we might want to avoid stemming. The list survives for the duration of the build and grows to be around 53,700 elements long.SearchChinese.steminternally usesword in self.latin_termsto check membership, and hence we get appalling performance as list membership testing isWith sphinx-doc/sphinx#12995,
IndexBuilder.feed()'only' takes 24% of build duration. The slowness is almost entirely due tojieba, as on a build without it installed the method takes 1.3%..feed()duration@mattwang44 -- do you know of any better Chinese segmenters than Jieba? It hasn't been released for four years, and is quite slow.
A
AA-Turner commentedon Oct 10, 2024
python/cpython#125264 looks to use PyStemmer more generally.
A
mattwang44 commentedon Oct 10, 2024
I'm not an expert but I believe Jieba still dominates this domain.
I know two packages that are tailored for zh_TW:
For zh_CN, there's a benchmark repo shows that https://github.com/tsroten/pynlpir is slightly faster than jieba.
I wonder what role does the tokenizer play in the Sphinx? Does it affect the performance of text searching (from the search bar on the doc)?
AA-Turner commentedon Oct 10, 2024
I think without it the search is worse (original PR sphinx-doc/sphinx#2170). You could try building a copy of the zh-TW translation with and without Jieba installed and see if the search results for a chinese term are meaningfully different?
A
AA-Turner commentedon Oct 10, 2024
Latest times below. Prior to Sphinx 8.1, zh-TW took 1h 28m 16s. Now it takes 5m 41s, which is a pretty decent improvement (~15.5x). Estimated time for a full rebuild loop is now
60180 minutes, I'll keep an eye on it.A
Build times (HTML only)
AA-Turner commentedon Oct 11, 2024
There reamain improvements to be made (e.g. speeding up or replacing Jieba), but the duration of the Chinese builds is no longer pathalogical. Thank you all for your support in improving this.
A
Build times (HTML only; Chinese)
mattwang44 commentedon Oct 11, 2024
Much appreciated @AA-Turner! Sphinx 8.1 is confirmed to reduce most local build time too.
We've been suffering from this issue for too long. This patch is the savior!