[derek73]

Part of #754, sub-phase 4i-d, the optional last PR of the Lexicon surface work. Three decisions were taken on the proposal for this PR: report the files that are on (not just the chosen ones), apply the same rule to a caller's own add/remove calls, and treat the Ph. D. question below separately.

What changes

Reading.languages (Token.reading and Ambiguity.alternative) now names the files whose reading of that kind is on in the parse's own Lexicon. Until now it named the files on by default, whatever choice of languages built the parser.

• The default lexicon (Lexicon.default(), no choice, no add/remove calls): nothing changes. None of the 6,531 committed corpus names move. Under Lexicon.empty(), 21 names move: the Ph. D. pair goes from en to ().
• Plain choice: names the chosen files plus every other file whose reading the choice left on. Under pt, de names pt, while van keeps nl.
• Strict choice: names the chosen files and native-script files only; mul and und drop out.
• A caller's add/remove calls:
  • A reading the caller adds of a kind no file holds for that word names no file, as before.
  • Adding a reading of a kind a file holds off by default switches that file's reading on, and the file is then named. Lexicon.default().add("st", kind="generation") names cs hr sk sl for Jiří Dienstbier st., where it named none.
  • A remove stops naming the file whose reading it took out.

Only languages moves, across the committed corpora (master 854087be against this branch):

Choice	Names that move	Examples
pt	309	de from es fr it nl pt to pt; E in ANA SILVA E SOUZA from it pt to pt
cs	11	Mgr. from en fr to cs; st. and ml. from () to cs
es	299	de, la, del and san to es
strict en	109	Ma. from en fil to en; Baron from de en fr nl to en
strict pt	337	21 of them the Ph. D. pair, from en to ()

How

• New lexicon slot: Lexicon gains a private _files slot, a _lexicon._ReadingFiles.
  • There is one per base, add/remove log and choice, shared between equal lexicons through an lru_cache.
  • It is hashed by identity, and never compared, hashed as part of the Lexicon, or pickled.
• Memos: assemble's two memos, _languages and _part_languages, take that slot as a fourth key, so a cache hit still costs no frame.
• Lookup by case:
  • The default lexicon still answers through languages_of.
  • A choice with no add/remove calls answers from the word's own rows under choose. That is exact, because a choice's preference is decided per word.
  • A lexicon with add/remove calls, or an old pickle's snapshot base, answers from the pool already compiled when the lexicon was built.
• Load cost: a first version read the whole pool on a cache miss. That added about 2 ms to the first parse that meets a listed word (parser_for(locales.PT): 0.23 ms to 2.5 ms). The per-word path removes it.
• Pickles: nothing new is stored on ParsedName, Token or Reading, so no ADDED_IN marker is needed. Old Lexicon and Parser pickles rebuild the slot on load.

Docs

• A new decisions/vocabulary-model 2026-11-02 sub-phase 4i-d entry. It records the three decisions and what each declined, and marks the 3b entry's "does not follow a choice" bullet superseded.
• The RFC marks its 4i-d row and status as built.
• The 3.0 Reading.languages release-log bullet gains the choice sentence. It stays one bullet, since the field itself is new in 3.0.
• The Reading.languages docstring and docs/usage.rst are updated.
• docs/locales.rst replaces its "names no file" paragraph with a doctest (ml. and Mgr. under locales.CS).
• AGENTS.md's index, layering and pickling notes mention the new slot.

Follow-up (not in this PR)

Ph. D. is merged by a fixed pattern whatever the vocabulary (decisions/phd-merge). So under parser_for(locales.PT, strict=True) or Lexicon.empty() it still reads as a credential with source vocabulary, while the joined PhD reads as the family name. It now names no file. An issue is drafted, not yet filed, asking whether the parse or the source should change.

Checks

• uv run pytest, uv run pytest -n0 -m serial, uv run mypy and uv run ruff check pass.
• sphinx-build -b doctest passes (229 passed), and so does python -m doctest README.rst.
• The suite passes without namedivider and on --python 3.15.
• tests/v2/test_snapshot.py is unchanged.
• tools/differential/cldr_agreement.py matches both records (JA not measured).
• Frames (tools/perf/call_count.py --against 854087be): the reference row goes from 311.48 to 311.72, all of it first-parse cache misses. A second pass reads 311.0 on both trees, and _CALL_BASELINE is unchanged.
• A check over nine lexicon configurations found the fast per-word lookups agreeing with the whole-pool answer on every corpus name.

🤖 Generated with Claude Code