Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ Latin-script input is never touched by any of this: "Kim Min-jun" is genuinely o

### Vocabulary (`nameparser/vocab/`) and the configuration layer (`nameparser/config/`)

**The default vocabulary is written per word, in TOML, and compiled.** `nameparser/vocab/<tag>.toml` holds one file per BCP 47 language tag, each `[[word]]` entry a `text` and its readings keyed by kind. What a file means, how an entry is built, the kinds and properties with the set each compiles into, exclusions, bearer counts and the steps for adding a word are in `nameparser/vocab/README.md`, the reference for anyone editing the TOML; `tests/v2/test_vocab.py` holds its kind, property, derived-marker and reserved-name tables to the names `build.py` accepts and the sets it compiles, so read it rather than restating it here. The parser never reads the TOML: `tools/vocab/build.py` runs the build checks that replaced the modules' import-time asserts (its docstring numbers them), turns every reading into an index row with its file's tag, and writes the rows as the committed reading index `nameparser/config/_index.py` (#753, phase 3a) and, compiled, as the committed `nameparser/config/_compiled.py`, frozenset literals one entry per line; the build never reads `_index.py` back. A file a shipped pack reads may also hold a `[rules]` table, each key a `Policy` field (`patronymic_rules`, `name_order`, `two_family_names`), which the build checks against its `RULE_KEYS` and `PACK_RULES` and writes as `nameparser/config/_rules.py`; `_locale.rules_patch(tag)` turns a file's table into the `PolicyPatch` RU and TR_AZ carry, from `ru.toml` and `tr.toml` (#754, 4f; decisions/vocabulary-model's 2026-10-29 4f entry). The compiler is `nameparser/_reading_index.py`'s `compile_readings`, with the kind and property tables it reads (`CATEGORY`, `BEHAVIORAL`, `GATED_WORDS`), so the build and a choice of languages compile alike: `Locale.languages` (3.0, phase 3b) is a choice of BCP 47 tags, which `parser_for` unions across its packs and applies as a delta against the default's compile, `choose` there selecting the readings (a chosen file's `default = false` ones on; on a word a chosen file holds a vocabulary reading of, other files' vocabulary readings off, save `mul.toml`'s and `und.toml`'s; a chosen conjunction's `may_be_initial` dropped), and `changed_words` deriving a choice's deviation surface, which tests/v2/test_locales.py's gate reads (decisions/vocabulary-model, 2026-10-29 3b); the build keeps the checks and loads that module by path, as it loads `_lexicon.py`, and `tests/v2/test_reading_index.py` holds that the index compiled with no choice is `_compiled.py` exactly. `readings_of(text)` there is the run-time reader of the index, every reading of a word with its file's tag, loaded on first call; `languages_of(text, categories)` is the reader a parse calls, from assemble, for `Reading.languages` (3.0), so the first parse that reads a word from its `Lexicon`'s vocabulary loads the index. Each `config/` module re-exports its constants from `_compiled.py` (`TITLES = _compiled.TITLES`) under its autodoc docstring, so the public names and the docs did not move. `tests/v2/test_vocab.py` fails when `_compiled.py` is stale, and `tests/v2/test_reading_index.py` when the index is, each naming the refresh command, and `test_vocab.py` gives every build check a mutation. The model, the phasing and what was declined: decisions/vocabulary-model.
**The default vocabulary is written per word, in TOML, and compiled.** `nameparser/vocab/<tag>.toml` holds one file per BCP 47 language tag, each `[[word]]` entry a `text` and its readings keyed by kind. What a file means, how an entry is built, the kinds and properties with the set each compiles into, exclusions, bearer counts and the steps for adding a word are in `nameparser/vocab/README.md`, the reference for anyone editing the TOML; `tests/v2/test_vocab.py` holds its kind, property, derived-marker and reserved-name tables to the names `build.py` accepts and the sets it compiles, so read it rather than restating it here. The parser never reads the TOML: `tools/vocab/build.py` runs the build checks that replaced the modules' import-time asserts (its docstring numbers them), turns every reading into an index row with its file's tag, and writes the rows as the committed reading index `nameparser/config/_index.py` (#753, phase 3a) and, compiled, as the committed `nameparser/config/_compiled.py`, frozenset literals one entry per line; the build never reads `_index.py` back. A file a shipped pack reads may also hold a `[rules]` table, each key a `Policy` field (`patronymic_rules`, `name_order`, `two_family_names`, `two_given_names`), which the build checks against its `RULE_KEYS` and `PACK_RULES` and writes as `nameparser/config/_rules.py`; `_locale.rules_patch(tag)` turns a file's table into the `PolicyPatch` RU and TR_AZ carry, from `ru.toml` and `tr.toml` (#754, 4f; decisions/vocabulary-model's 2026-10-29 4f entry). The compiler is `nameparser/_reading_index.py`'s `compile_readings`, with the kind and property tables it reads (`CATEGORY`, `BEHAVIORAL`, `GATED_WORDS`), so the build and a choice of languages compile alike: `Locale.languages` (3.0, phase 3b) is a choice of BCP 47 tags, which `parser_for` unions across its packs and applies as a delta against the default's compile, `choose` there selecting the readings (a chosen file's `default = false` ones on; on a word a chosen file holds a vocabulary reading of, other files' vocabulary readings off, save `mul.toml`'s and `und.toml`'s; a chosen conjunction's `may_be_initial` dropped), and `changed_words` deriving a choice's deviation surface, which tests/v2/test_locales.py's gate reads (decisions/vocabulary-model, 2026-10-29 3b); the build keeps the checks and loads that module by path, as it loads `_lexicon.py`, and `tests/v2/test_reading_index.py` holds that the index compiled with no choice is `_compiled.py` exactly. `readings_of(text)` there is the run-time reader of the index, every reading of a word with its file's tag, loaded on first call; `languages_of(text, categories)` is the reader a parse calls, from assemble, for `Reading.languages` (3.0), so the first parse that reads a word from its `Lexicon`'s vocabulary loads the index. Each `config/` module re-exports its constants from `_compiled.py` (`TITLES = _compiled.TITLES`) under its autodoc docstring, so the public names and the docs did not move. `tests/v2/test_vocab.py` fails when `_compiled.py` is stale, and `tests/v2/test_reading_index.py` when the index is, each naming the refresh command, and `test_vocab.py` gives every build check a mutation. The model, the phasing and what was declined: decisions/vocabulary-model.

The `config/` SETS are frozen since 2.2 (#293): there is no `.add()`/`.remove()` on any of them, so a default word list is changed at runtime by configuring a `Lexicon`, never by editing the constant, and the shipped default is changed by editing the TOML and rebuilding. **Neither dict was frozen, and `CAPITALIZATION_EXCEPTIONS` is not covered by anything else either.** `REGEXES` is a compiled-pattern table rather than vocabulary and was never in #293's scope; `CAPITALIZATION_EXCEPTIONS` is vocabulary-shaped and is a decided, in-scope exemption. So the hazard the freeze closes is still live for it: `CAPITALIZATION_EXCEPTIONS['dclinpsy'] = 'DClinPsy'` after the first parse misses the cached `Lexicon.default()`. Same advice — configure the object (`dataclasses.replace(lexicon, capitalization_exceptions=...)`). `tests/v2/test_contracts.py::test_every_vocabulary_constant_is_frozen` names both dicts as explicit exemptions rather than letting its `isinstance` filter drop them.

Expand Down
4 changes: 3 additions & 1 deletion docs/concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,9 @@ word is the given name, the last is the family name, and anything
between them is the middle name. ``name_order``, an explicit comma
and ``script_orders`` change what "first" and "last" mean here, as
does the opt-in ``two_family_names``, which takes the last two name
words as the family (:ref:`two-family-names`).
words as the family (:ref:`two-family-names`), and the opt-in
``two_given_names``, which gives the given name the middle name beside
it (:ref:`two-given-names`).
``script_orders`` covers the East Asian writing that settles its own
order: a name wholly in Han or hangul, or a Japanese name in kanji and
kana — any of them, hiragana alone included — unless it is katakana
Expand Down
59 changes: 59 additions & 0 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -694,6 +694,12 @@ listed below.
- Reads the last two name words as the family name, for data
where people carry two family names (Spain, most of Latin
America). Defaults to ``False``. See :ref:`two-family-names`.
* - ``two_given_names``
- ``bool``
- Reads the middle name beside the given name as part of it, for
data where people carry a compound given name and no middle
name (``María José``). Defaults to ``False``. See
:ref:`two-given-names`.
* - ``nickname_delimiters``
- ``frozenset[tuple[str, str]]``
- Routes content enclosed by these delimiter pairs to
Expand Down Expand Up @@ -1017,6 +1023,59 @@ breaks 8,234 names of 20,000 and mends 265. The parser
reports nothing for the names the switch moves: the shape it reads is
the same either way.

.. _two-given-names:

Names with a compound given name
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A Spanish or Portuguese given name is often two names, such as María
José or Juan Carlos, and its bearer has no middle name. The default
reads the second as a middle name, the English convention:

.. doctest::

>>> carlos = parse("Juan Carlos Pérez")
>>> carlos.given, carlos.middle
('Juan', 'Carlos')

``two_given_names=True`` gives the given name the middle name beside
it. Any middle names past it stay middle names, a family comma leaves
the part after it to the switch, and with ``two_family_names`` on too
the family takes its word first, so a name of four words reads two and
two:

.. doctest::

>>> two_given = Parser(policy=Policy(two_given_names=True))
>>> two_given.parse("Juan Carlos Pérez").given
'Juan Carlos'
>>> two_given.parse("Pérez, Juan Carlos Antonio").middle
'Antonio'
>>> both = Parser(policy=Policy(two_family_names=True,
... two_given_names=True))
>>> mj = both.parse("María José García López")
>>> mj.given, mj.middle, mj.family
('María José', '', 'García López')
>>> both.parse("Gabriel García Márquez").given
'Gabriel'

A given name a particle completes reads whole where a second particle
starts the family (``"María del Carmen de la Vega"`` gives given
``María del Carmen``); where the particle's words run to the end of
the name, as in ``"Maria do Carmo Silva"``, they stay the family with
the switch on or off, the same shape as ``"João da Silva Santos"``.

Turn it on only for data written this way: under it every name whose
second given word is a middle name gains it in the given name
(``"John Quincy Adams"`` gives given ``John Quincy``), and people
filling a form that has a middle-name box split their names the other
way. Among the UK company register's 6,089 Spanish filers, 31.8% of
names agree in every box by default and 15.7% with this switch on;
with ``two_family_names`` on, 80.2% without this switch and 55.0%
with it (measured 2026-10-30). ``middle`` keeps its meaning either way,
the given part's words after the given name; the switch moves no word
anywhere else and reports nothing.

.. _east-asian-defaults:

East Asian defaults, and turning them off
Expand Down
Loading
Loading