[derek73]

Completes 3.0 sub-phase 4a (#754): each token's Token.reading, and each report's Ambiguity.alternative, now names the shipped language files that list the word the way it was read. The lookup uses 3a's reading index (#753).

>>> [(t.text, t.reading.languages) for t in parse("Dr. Juan de la Vega III").tokens]
[('Dr.', ('mul',)), ('Juan', ()), ('de', ('es', 'fr', 'it', 'nl', 'pt')), ('la', ('es', 'fr', 'it')), ('Vega', ()), ('III', ('mul',))]

What changed

• Reading.languages: tuple[str, ...] = (), compare=False, metadata={ADDED_IN: "3.0.0"}.
  • Reading shipped in 2.7, so a Reading pickled by 2.7 or 2.8 loads with (). The new fixture tests/v2/data/parsedname_v28_alternative.pickle was made by the released 2.8.0 wheel.
  • A renamed field, or a shipped field missing beside this one, is still refused (_REFUSED_SKEWS).
  • The constructor takes any iterable of str. It refuses, with TypeError, a bare str, a mapping, a non-iterable and a non-str tag.
  • The repr names the files where there are any: Reading('particle' as 'family', 'vocabulary', in es fr it nl pt).
• Filled at assemble (_pipeline/_assemble.py) for a vocabulary or declared reading only, through _reading_index.languages_of(text, categories).
  • A reading kind matches index rows by category (_CATEGORIES). Every suffix kind matches the word's suffix rows, since a word has one suffix kind.
  • A word is looked up by its normalized form, then with its periods removed (M.A, Lt.Gen. as ltgen).
  • A name reading takes the rows listing the word as a name of the field it lands in: family for a family or maiden word, given for a given or middle word (a middle that middle_as_family folds included), and bound given names in either.
  • A credential split across a space is looked up whole (Ph. D.), passing over a nickname written between its words.
  • A word joining listed words is looked up by its parts (Lt.Gov., M.D./Ph.D., MD;), never dividing at a combining mark (प्रोफे.डॉ. → hi, mr). This happens only where the parse's own Lexicon doesn't list the word whole, so a caller's own zzq-dr title names no file.
  • Unions over parts, and an alternative's union over the words its report names, are sorted.
• Layering: _pipeline/_assemble.py may import nameparser._reading_index (tests/v2/test_layering.py).
• Carried in: cherry-pick of 84ac249e (4a's coverage fix to _opens_its_segment), kept as its own commit.

Sample readings

word	reading
particle de (Dr. Juan de la Vega III)	particle, vocabulary, es fr it nl pt
credential PhD (John Smith PhD)	credential, vocabulary, en
CJK surname 김 (김민준)	name, vocabulary, (): the Korean surname list stays outside the per-word files
honorific 先生 (王小明先生)	post-nominal, vocabulary, ja zh
unlisted Juan	name, position, ()
alternative of MA (JOHN SMITH MA)	name as family, declared, en (fil lists ma only as a given name)

Over the snapshot's 6,360 names (20,804 tokens), 3,935 readings are vocabulary or declared, and 3,750 of them name a file. Of the 185 that name none, 157 are Korean surnames split off by the list, and 28 are declared names read into a field no file lists them in (van as a family name, for example; nl lists it as given).

Costs

Measured 2026-10-29 on Python 3.11.17 against bd02d68f.

• First parse. In a fresh process with bytecode cached, as an installed wheel has it, the first parse that reads a listed word takes 1.20–1.22 ms, against 0.22 ms on base (medians of 21 processes, three runs). That is about 1.0 ms once per process: the index import plus the first lookups. A rerun on a quieter machine gave 0.78 ms. Without bytecode (PYTHONDONTWRITEBYTECODE=1, as in some containers) it is about 16 ms.
  • Which parse triggers the load is decided by the parse's own Lexicon. 김민준 and a caller's own title load the index; John Smith does not.
• Steady per-parse: 78.6–79.0 µs against 78.3–79.4 µs on base, which is within noise.
• Frames: tools/perf/call_count.py gives 311.00 → 311.48 on 3.11.
  • All of the rise is the first of the 50 reference parses (335 frames against 311). After it, both trees count 311.0.
  • On 3.15 the tree gives 292.40, inside the band.
  • Bounded lru_cache memos keep the lookups off the frame count, since a hit runs in C. _CALL_BASELINE is not moved.
  • One frame per parse remains, for a word read by its parts (Lt.Gov., M.D./Ph.D.). It checks that the parse's own Lexicon doesn't list the word whole, and since that depends on the lexicon it can't be memoized on the text.
• Serial benchmark: the frame-budget test passes. Several wall-clock scaling tests failed on this machine, a different one on each run. Two runs of the same module on bd02d68f failed the same way (delimiter_pairs, then title_chain), so this is load noise.

No parse moves

No role, tag or alternative kind/role/source changes. languages is not compared, and the snapshot does not record readings, so the committed snapshot is byte-identical and test_snapshot passes unchanged. One 4a Token.reading moves, on a shape the snapshot does not hold: the second word of a split credential with a nickname between its words (D. in John Smith Ph. "Bob" D.) now takes its first word's credential/vocabulary reading, where 4a gave credential/position. The decisions entry records it.

Not done here

• Kind for Jr/Sr (4b). This lookup does not make it free. The kind a token takes is named by _vocab.suffix_kind, which also names Ambiguity.alternative. Reading the kind from the index at assemble alone would split the two, and reading it inside suffix_kind would move alternatives. The strict xfails stand.
• Under 3b's choice of languages, languages_of's on-by-default filter, and the memo's key, are the places to change. A note in the session notes tells the 3b lane.

Docs (same-PR amendment)

• RFC: runtime section, Phase 4 status, the Token.reading item, work-order row 4a, and the Lexicon open question.
• decisions/vocabulary-model: a new 2026-10-29 "4a's follow-up" entry with a SUPERSEDES: line for decisions/A1's 2.7 decline of languages, and a new Standing line. A1's entry head carries the partial-supersede marker. The earlier 3a and 4a entries are left as written; the new entry says what changed.
• Also updated: root AGENTS.md (the index reader, layering), nameparser/_pipeline/AGENTS.md, docs/usage.rst (a doctest), and a release-log bullet.

Checks run: uv run pytest (also on 3.15), uv run pytest -n0 -m serial, uv run mypy, uv run ruff check, sphinx-build -b doctest. Reviewed by a design-docs review and a code review; both rounds' findings are answered in the second commit.

Refs #754, #753.

🤖 Generated with Claude Code