Read the vocabulary sets through Lexicon._sets (#754, 4i-a)

docs/design/decisions/vocabulary-model.md

@@ -486,3 +486,10 @@ Open: #913 (the title words phase 2j left, read as titles though borne as names)
   6. **Migration.** The old spellings (`add`/`remove` with a set keyword, `Lexicon(...)` and `dataclasses.replace` with one, a set attribute read) fail loudly with a per-word translation, the one the pickle load uses.
   7. **Work order.** 4i-a, the pipeline reading the sets through one internal object; 4i-b, the break; 4i-c, the recorded choice; 4i-d, `Reading.languages` following the choice, optional; none moves a default parse.
   - Declined: letting the caller's reading win where it clashes with a reading a later choice turns on (the clash is refused at construction, and one rule at both points is the one a caller can predict); an eager default base, holding the index rows in `Lexicon.default()` (it would reverse the 2026-10-29 lazy load for every caller, about 2 ms cached and 18 to 20 ms uncached on the first use); refusing every 2.x `Lexicon` state, pickle option (a) (the default and the 2.8 fixtures translate back exactly, and the commonest customization, `Lexicon.default().add(titles={"dean"})`, does with a word in both `titles` and `titles_ambiguous` compared as in `titles` alone, the two spellings parsing alike). The alternatives the proposal weighed for its other items, each recorded there with its reason, are declined with them: a positional `add(word, kind)`, a reading object, new method names, publishing `IndexReading`, a predicate per category, recording no choice, a `Parser` field for it, and pickling the base by reference.
+- **2026-11-02 #754, sub-phase 4i-a — THE PARSE READS THE VOCABULARY SETS THROUGH ONE INTERNAL OBJECT, `Lexicon._sets`, NOT THE PUBLIC FIELDS; THE THREE TOOLS MOVE TO IT; NOTHING PUBLIC CHANGES AND NO PARSE MOVES.** Carries out item 7's first PR of the 2026-11-02 #754, the `Lexicon` surface entry.
+  - **What.** `_lexicon._LexiconSets`, a frozen, slotted object with one frozenset per `_VOCAB_FIELDS` entry: the 20 fields 4i-b makes internal, and `surnames` and `given_names`, which stay public but sit in the rosters beside them (`_assemble._listed_whole` walks all of `_VOCAB_FIELDS`, and `_vocab.in_any_wordlist` holds `surnames`; `claimed_as_non_name` holds neither), so that a roster is read off one object. Each `Lexicon` builds one in `__post_init__`, from its normalized fields by keyword (tests/v2/test_lexicon.py holds its field names to `_VOCAB_FIELDS`, in order, so a field added to one and not the other fails), into a slot `_sets` inherited from a plain base class (`_HoldsSets`): a dataclass field, even `init=False`, would have changed `dataclasses.fields(Lexicon)`, the pickled state and the 2.8 pickles' layout check. It is not compared, hashed or pickled; a load, `dataclasses.replace` and `|` rebuild it. Every library module but `_lexicon` reads `lexicon._sets.titles` where it read `lexicon.titles`; a membership test stays a C-level attribute read, one more attribute load and no frame.
+  - **Measured: the reads,** on this branch over `b55d4d33`, by the 2026-11-01 entry's recipe (an attribute load of a set field, `surnames` and `given_names` left out): its 111 library reads in 12 files are 8 of `_lexicon`'s own, which stay, 2 `TailRead.titles` reads in `_assign` and `_group`, no lexicon's, and 101 that moved, in 11 files. Moved beside them: 5 reads of `surnames` and `given_names`, and 5 `getattr` reads over a roster (the three above and `_parser._apply_choice`'s two). The tools' 16 `Lexicon` reads moved to `_sets`, none to the index: `local_agreement` reads a pack parser's lexicon, which the shipped index does not describe, and `name_usage` reads `suffix_acronyms_ambiguous`, a marker the compile derives and no index row states. The 17th is `local_corpora`'s counter.
+  - **Measured: the frames,** Python 3.11.17, the benchmark's counter over 50 names with a four-digit counter, `b55d4d33` against this branch: the reference row 311.82 and 311.82 (`uv run python tools/perf/call_count.py --against b55d4d33`); by the same counter, `Smith{i:04d}, John` 168.0 and 168.0, `de la Vega{i:04d}, Juan Q., Jr.` 334.16 and 334.16, `John Smith{i:04d}` 158.0 and 158.0. Wall clock over 6,000 names moved within the run-to-run spread (96 to 98 µs a name on both trees). `_CALL_BASELINE` is unchanged.
+  - **The guard.** tests/v2/test_layering.py walks every library module but `_lexicon`, and every file under tools/, and refuses a vocabulary field read off a lexicon, one test per `_VOCAB_FIELDS` entry: in the library only a read through `<x>._sets`, or through a name `sets` that every binding in the module assigns from a `_sets`, passes, `TailRead`'s `read.titles` named as no lexicon, and a `getattr` of a lexicon, or of a field named as a constant off anything but a `_sets`, fails; in the tools, whose counters reuse the field names, a read off `lexicon`, `lex`, `<x>.lexicon`, `Lexicon.default()` or a name any binding gives one of those (an assignment, a walrus, a `Lexicon` annotation) fails, the AGENTS.md recipe's `L = Parser().lexicon` among them. Not followed, and stated in the check's docstring: a lexicon handed through a container, a call's return or an attribute other than `.lexicon`, and aliases per scope rather than per module. Its negative controls: a synthetic read in each spelling, library and tool, per field, must be refused, each binding form of `sets` and of an alias has a row, and `_UNGUARDED_READS` records what the check refuses on today's sources with `._sets` taken out again (114 in the library: 106 attribute loads, 5 roster `getattr`s, 3 of them through the `sets` the undoing makes a lexicon, and 3 `sets =` bindings; 16 in the tools, of 40 files). Mutated on the built tree, one `lex._sets.titles_ambiguous` in `_classify` written back as `lex.titles_ambiguous` failed exactly `test_no_reader_takes_a_vocabulary_field_off_a_lexicon[titles_ambiguous]`.
+  - **How 4i-b plugs in.** 4i-b changes how `_sets` is filled and nothing that reads it: from the lexicon's readings, compiled by `compile_readings`, rather than from its public fields; the default's from nameparser/config/_compiled.py, as `_default_lexicon` reads it today, so `parse("John Smith")` still loads no index (the base stays lazy, the 2026-11-02 entry's item 4). The derived markers the views will read, a listed suffix word's kind and the patronymic particles (the RFC's item 1), join `_LexiconSets` as attributes, and `_vocab.suffix_kind` and classify's particle kind read them there in place of the index. With `dataclasses.fields(Lexicon)` changing in 4i-b anyway, `_sets` may become a private field then, the base class going. `_parser._apply_choice` reads `_sets` but still writes the public fields through `dataclasses.replace`; 4i-b gives it the readings to write until 4i-c retires it. What 4i-b also breaks, outside the readers the guard holds: the tests' 157 public-field reads (the 2026-11-01 count), and AGENTS.md's recipe for the titles in no suffix set, `L.titles - L.suffix_acronyms - L.suffix_words` on `L = Parser().lexicon`, which reads public fields and is rewritten then over `readings_of` or `_sets`.
+  - The default does not move: tests/v2/test_snapshot.py passes unchanged and tests/v2/test_cldr_agreement.py with it; the suite passes on Python 3.11 and 3.15 without the `ja` extra.

docs/design/vocabulary-model-rfc.md

@@ -463,7 +463,7 @@ How a call reaches a pack's pattern is open, and each answer costs something: `P
 
 ### The `Lexicon` surface: configuring by reading
 
-Status: **decided 2026-11-02 as recommended below (decisions/vocabulary-model's 2026-11-02 #754, the `Lexicon` surface entry), item 1's title warning on 2026-11-01 (decisions/vocabulary-model's 2026-11-01 #754, the title warning entry); nothing is built.** What is decided is the direction (2026-10-28): a caller configures vocabulary by reading, `Lexicon` derives the sets and gates the build derives, and the compiled sets become internal (on 2026-11-01, 20 of `Lexicon`'s 23 public fields: the 19 of `_lexicon._COMPILED_FIELDS` and the derived `particles_ambiguous`; item 3 keeps the other three), with no 2.x warning (2026-10-29). This section proposes the API that direction leaves open, each question with the options weighed and one recommendation; decisions/vocabulary-model's 2026-11-01 #754, the `Lexicon` surface proposed entry records the proposal and its measurements, and decisions/vocabulary-model's 2026-11-02 #754, the `Lexicon` surface entry the decision, each recommendation accepted and the alternatives below declined. Where this section says "proposed" or "recommended", read "decided"; the wording is kept as the record of what was weighed. Every parse below was run on master `a3b1f3e1` (Python 3.11.17) through a throwaway prototype that compiles a lexicon's readings with the shipped `compile_readings` over `choose(index(), chosen)`, the whole index with its readings off by default, as item 4 has the default hold them, the caller's edits applied after the choice, and folds the result into today's `Lexicon`, so the real parser reads it; the prototype is not committed.
+Status: **decided 2026-11-02 as recommended below (decisions/vocabulary-model's 2026-11-02 #754, the `Lexicon` surface entry), item 1's title warning on 2026-11-01 (decisions/vocabulary-model's 2026-11-01 #754, the title warning entry); 4i-a is built (decisions/vocabulary-model's 2026-11-02 #754, sub-phase 4i-a entry), the rest is not.** What is decided is the direction (2026-10-28): a caller configures vocabulary by reading, `Lexicon` derives the sets and gates the build derives, and the compiled sets become internal (on 2026-11-01, 20 of `Lexicon`'s 23 public fields: the 19 of `_lexicon._COMPILED_FIELDS` and the derived `particles_ambiguous`; item 3 keeps the other three), with no 2.x warning (2026-10-29). This section proposes the API that direction leaves open, each question with the options weighed and one recommendation; decisions/vocabulary-model's 2026-11-01 #754, the `Lexicon` surface proposed entry records the proposal and its measurements, and decisions/vocabulary-model's 2026-11-02 #754, the `Lexicon` surface entry the decision, each recommendation accepted and the alternatives below declined. Where this section says "proposed" or "recommended", read "decided"; the wording is kept as the record of what was weighed. Every parse below was run on master `a3b1f3e1` (Python 3.11.17) through a throwaway prototype that compiles a lexicon's readings with the shipped `compile_readings` over `choose(index(), chosen)`, the whole index with its readings off by default, as item 4 has the default hold them, the caller's edits applied after the choice, and folds the result into today's `Lexicon`, so the real parser reads it; the prototype is not committed.
 
 **1. Configuring: `add` and `remove` take words and a reading kind.** Proposed:
 
@@ -559,7 +559,7 @@ All three were checked on a frozen, slotted throwaway dataclass on 2026-11-01. T
 
 | Sub-phase | What | Needs |
 |---|---|---|
-| 4i-a | The pipeline reads the compiled sets through one internal object, not `Lexicon`'s public fields; the three tools move to it or to the index. No public change; a frozenset test stays a C-level attribute read, so the frame count holds (`_CALL_BASELINE`) | — |
+| 4i-a | The pipeline reads the compiled sets through one internal object, not `Lexicon`'s public fields; the three tools move to it or to the index. No public change; a frozenset test stays a C-level attribute read, so the frame count holds (`_CALL_BASELINE`). Landed 2026-11-02 (decisions/vocabulary-model's 2026-11-02 #754, sub-phase 4i-a entry) | — |
 | 4i-b | The break: `Lexicon` holds readings, `add`/`remove` by kind, `readings_of` and `LexiconReading`, the narrowed constructor, the three refusals with their translation, the per-kind repr, the warning on a title reading made phrase-only by a `given` reading, the views reading the lexicon's kinds with rules/R8 and rules/R9 amended, the lazy base, the 2.x pickle translation; the tests and docs moved, `docs/migrate.rst`'s table, a release-log bullet | 4i-a |
 | 4i-c | The choice recorded on the `Lexicon`: chained choices union, a caller's readings applied after the choice, the strict mismatch raised, `|` refusing two choices, a pack's fragment joined under the caller's lexicon, the compile and its refusals at `parser_for`, the underscore choice parameters validated, `_apply_choice` retired; the chained-choices entry and 3j's dropped-listing clause superseded | 4i-b |
 | 4i-d | Optional: `Reading.languages` follows the choice | 4i-c |
@@ -578,7 +578,7 @@ Each sub-phase is one PR, measured on its own base as phase 2's and phase 3's ar
 | 4f | `[rules]` compiled into a `Locale`'s policy patch; RU and TR_AZ rebuilt from their files. Landed (decisions/vocabulary-model's 2026-10-29 sub-phase 4f entry) | 3b |
 | 4g | the compound given name, a rule kind (#704): built 2026-10-30, `Policy.two_given_names` (rules/O7) | 4f |
 | 4h | the shape inventory per order (#469), starting with the title between a family and a given name under `FAMILY_FIRST`. Landed as an inventory, no reading moved: the arrangements each order is written in, listed with what each reads (decisions/vocabulary-model's 2026-10-30 #754, sub-phase 4h entry); the title between, and bare after, under a family-first order stays a name word, #938 declined on a survey of real names, which write the title in front (decisions/H5's 2026-10-30 #938 entry) | — |
-| 4i | the `Lexicon` surface: configuring by reading, the compiled sets internal, and the choice recorded on the `Lexicon`. Decided 2026-11-02, to be built in four PRs (4i-a to 4i-d, "The `Lexicon` surface: configuring by reading" above) | 3b, 3j |
+| 4i | the `Lexicon` surface: configuring by reading, the compiled sets internal, and the choice recorded on the `Lexicon`. Decided 2026-11-02, to be built in four PRs, 4i-a landed (4i-a to 4i-d, "The `Lexicon` surface: configuring by reading" above) | 3b, 3j |
 
 **Interleaved with phase 3** (its sub-phases 3a to 3j, above): 4a can start beside 3a, its `languages` waiting for 3a's index; 4d needs nothing of phase 3, and 4b and 4c need 3a's index only for a listed word's kind, a suffix word's and a particle's; they unblock its packs, `cs`'s generations (3f) and `pt-BR`'s agnomens (3h) on 4b, `es` and `pt` (3g) on 4d and 4g; 4f lands with or after 3b, since both decide what a shipped `Locale` holds; an `is` pack, which no phase 3 sub-phase lists, needs 3b, 4c and 4f. 3j, file-strict, needs 3b alone (3i, which name-strict would have read, is set aside), and 4e and 4h stand apart from both. So the order is 3a and 4a, then 4b to 4d with 3b and 3c (4b after 3a), then 4f, then the packs with 4g, with 4e at any point and 3j at any point after 3b.
 

nameparser/_lexicon.py

@@ -567,8 +567,68 @@ def _normpairs(
     return tuple(sorted(deduped.items()))
 
 
+@dataclass(frozen=True, slots=True, eq=False, repr=False)
+class _LexiconSets:
+    """The vocabulary sets a parse reads, held apart from the public
+    `Lexicon` fields: the pipeline, `_render`, `_parser` and the tools
+    read `lexicon._sets.titles`, never `lexicon.titles` (#754, 4i-a;
+    tests/v2/test_layering.py holds it). One attribute per
+    `_VOCAB_FIELDS` entry, in that order, each a frozenset, so a
+    membership test stays a C-level attribute read and costs no frame.
+
+    Built once per `Lexicon`, in its `__post_init__`, from the
+    lexicon's own fields, and never compared, hashed or pickled: a
+    `Lexicon`'s equality and pickle are its fields', and a load or a
+    `dataclasses.replace` builds this again. 4i-b, which makes the
+    compiled fields internal, changes only how it is filled: from the
+    lexicon's readings rather than from its public fields, the default
+    from nameparser/config/_compiled.py as today, so the reading index
+    stays unloaded on `parse("John Smith")`; the derived markers the
+    views will read (a listed suffix word's kind, the patronymic
+    particles) join it as attributes of their own. Nothing that reads
+    it moves then.
+
+    Not a mapping and no repr of its own: it holds whole word lists,
+    and a repr of them would be unbounded."""
+
+    titles: frozenset[str]
+    given_name_titles: frozenset[str]
+    suffix_acronyms: frozenset[str]
+    suffix_words: frozenset[str]
+    suffix_acronyms_ambiguous: frozenset[str]
+    particles: frozenset[str]
+    particles_ambiguous: frozenset[str]
+    conjunctions: frozenset[str]
+    conjunctions_ambiguous: frozenset[str]
+    bound_given_names: frozenset[str]
+    given_name_abbreviations: frozenset[str]
+    maiden_markers: frozenset[str]
+    surnames: frozenset[str]
+    honorific_tails: frozenset[str]
+    after_given_titles: frozenset[str]
+    suffix_words_ambiguous: frozenset[str]
+    agnomens: frozenset[str]
+    titles_ambiguous: frozenset[str]
+    title_modifiers: frozenset[str]
+    title_styles: frozenset[str]
+    title_phrases: frozenset[str]
+    given_names: frozenset[str]
+
+
+class _HoldsSets:
+    """`Lexicon`'s one slot outside its dataclass fields, holding its
+    `_LexiconSets`. A slot inherited from a plain base, because a field,
+    even one built `init=False`, would change `dataclasses.fields`, the
+    pickled state and the 2.8 pickles' layout check, and 4i-a changes
+    nothing public; `@dataclass(slots=True)` takes no slot of its own
+    beside its fields. Set once, by `Lexicon.__post_init__`."""
+
+    __slots__ = ("_sets",)
+    _sets: _LexiconSets
+
+
 @dataclass(frozen=True, slots=True)
-class Lexicon:
+class Lexicon(_HoldsSets):
     """The vocabulary a parser matches against: which words are
     titles, particles, suffixes, and so on. Immutable and hashable.
     Start from :meth:`default` (the shipped vocabulary) or
@@ -884,6 +944,9 @@ def __post_init__(self) -> None:
                 f"suffix_words_ambiguous exists to impose; drop it from "
                 f"suffix_acronyms or from suffix_words_ambiguous"
             )
+        # last, from the normalized fields: what the parse reads
+        object.__setattr__(self, "_sets", _LexiconSets(
+            **{name: getattr(self, name) for name in _VOCAB_FIELDS}))
 
     # -- constructors ----------------------------------------------------
 

nameparser/_parser.py

@@ -98,7 +98,7 @@ def __post_init__(self) -> None:
                 script.value
                 for script in self.policy.segment_scripts
                 if not any(_SCRIPT_MATCHERS[script](entry)
-                           for entry in self.lexicon.surnames))
+                           for entry in self.lexicon._sets.surnames))
             if uncovered:
                 names = ", ".join(uncovered)
                 one = len(uncovered) == 1
@@ -329,14 +329,14 @@ def _apply_choice(lexicon: Lexicon, languages: frozenset[str],
     fields, (unmasked, masks) = _choice_delta(
         _reading_index_selected(languages, strict), strict)
     updates: dict[str, object] = {
-        name: (getattr(lexicon, name) - gone) | joined
+        name: (getattr(lexicon._sets, name) - gone) | joined
         for name, (gone, joined) in fields.items()}
     # a word the choice drops from a base set leaves that set's markers
     # too, a caller's own mark of it included: marking a word the
     # vocabulary no longer holds is the orphan Lexicon refuses
     for marker, base, _ in _SUBSET_FIELDS:
         if base in fields and fields[base][0]:
-            kept = updates.get(marker, getattr(lexicon, marker))
+            kept = updates.get(marker, getattr(lexicon._sets, marker))
             updates[marker] = kept - fields[base][0]  # type: ignore[operator]
     if unmasked or masks:
         cap = {k: v for k, v in lexicon.capitalization_exceptions