Repository navigation
Vocabulary model and language packs (tracking) #747
Description
Goal. Describe each vocabulary word by its readings — what the word can be in a name (a generation suffix, a credential, a family particle, a salutation…) and the properties of each reading — instead of base sets plus marker subsets. Then build language packs on top: choosing a language settles the words that are ambiguous only because the default serves every language.
The default keeps working for names of any language, as today. A pack prefers its language, settling clashes where that language has a vocabulary reading; a strict pack also drops other languages' name readings.
Where things are
- The model and every decision so far: decisions/vocabulary-model
- The data:
nameparser/vocab/<language>.toml, compiled bytools/vocab/build.py(the parser never reads the TOML) - A reference for editing the TOML and a design overview of the later phases: the docs sub-issue below
Status
| Phase | What | State |
|---|---|---|
| 1 | Per-word TOML format, compiled to today's sets; no parse moves | done: #737, #744 (2.6) |
| 1, data | Classify the words still in und.toml by language and kind |
done: #765 (particles, #749), #769 and #773 (titles, #750), #778 and #780 (credentials and the rest, #751) |
| 2 | Ambiguity derived from clashing readings; the ambiguity gate extended to every reading kind | done: #752 (2.6–2.7, phases 2a–2m); title words left over are #913 |
| 3 | Language packs: a choice of languages turns on readings and prefers them | mostly done (3.0): 3a #941, 3b #945, 3c #946, 3j file-strict #986, packs ne #955, hi/bn #958, cs/sk/sl/hr #957 and #973, ga/gd/pt-BR #956, es/pt #964; 3i set aside. Open: an is pack, #345's transliterated honorifics, père (#684). See #753 |
| 4 | Order and shapes, the rules a pack can choose, the output model | mostly done (3.0): 4a Token.reading #942/#944, 4b generation/credentials #947, 4c patronymic #948, 4d surname2 #951, 4e render order #943, 4f [rules] #953, 4g two_given_names #954, 4h shape inventory #974, 4i the Lexicon surface by reading #989/#991/#992/#997. Open: particle phrases completing a given name (#704), render patterns. See #754 |
Classification (master af450cd8): every word is in the file of a language whose names it occurs in, except bon and dí, two particles of unrecorded origin left in und.toml; whether they stay is #766. Recompute with uv run python tools/vocab/query.py --file und.
Sub-issues track each piece. Related questions are linked under the phase they belong to.
Activity
Add docs/design/vocabulary-model-rfc.md, where the vocabulary model i…
Add docs/design/vocabulary-model-rfc.md, where the vocabulary model i…
Add docs/design/vocabulary-model-rfc.md, where the vocabulary model i…
Check 10 after review: run last, name the entry to move, refuse every…
Check 10, second review: place against the kept run, end on its own (#…
Check the vocabulary word order in the build (check 10) (#747) (#767)
Link zh Traditional and Simplified spellings with same_as (#747) (#790)
Standing block for decisions/vocabulary-model (#762) (#800)
John Smith, PrandProfesor Juan Pérezread the title as a given name: add French and Spanish professor titles #804