Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions AGENTS.md

Large diffs are not rendered by default.

23 changes: 15 additions & 8 deletions docs/design/decisions/vocabulary-model.md

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions docs/design/mechanisms.md
Original file line number Diff line number Diff line change
Expand Up @@ -332,11 +332,11 @@ a fork for all ambiguous particles on "Dr. Van Jr.".

**Problem shape.** Language-specific behavior needs a home that cannot drift from the core.

**Contract statement.** A locale pack is pure data — a choice of languages, a Lexicon fragment of ADDITIONS and a Policy patch — applied by parser_for, a shipped pack's patch compiled from its language file's `[rules]` table where one is written, each key the `Policy` field of its name: the packs' languages union into one choice, which selects among the default vocabulary's readings by the file each sits in, the fragments union onto the result, and the patches fold in order; packs contain no code paths of their own. A vocabulary pack's deviations are derived from the reading index, the words whose compiled readings its choice changes; a policy or segmentation pack declares its own in its docstring and `DEVIATES`.
**Contract statement.** A locale pack is pure data — a choice of languages, a Lexicon fragment of ADDITIONS and a Policy patch — applied by parser_for, a shipped pack's patch compiled from its language file's `[rules]` table where one is written, each key the `Policy` field of its name: the packs' languages union with the base's recorded choice into one choice, recorded on the `Lexicon`, which selects among the base's shipped readings by the file each sits in, the fragments join in pack order and the caller's lexicon after them, its own readings applying after the choice, and the patches fold in order; packs contain no code paths of their own. A vocabulary pack's deviations are derived from the reading index, the words whose compiled readings its choice changes; a policy or segmentation pack declares its own in its docstring and `DEVIATES`.

**How it works.** What is policy for every language (an order constant) is policy, not pack data; packs are lowercase modules exposing uppercase constants; a pack error is wrapped with the pack's code (rule D2). The fragment never removes a base entry; a choice may, and only by selection, never by a list of removals: a chosen language's `default = false` readings join; on a word a chosen language holds a vocabulary reading of, other languages' vocabulary readings leave, save `mul.toml`'s; and a chosen language's conjunction loses `may_be_initial`, its clash with the initial being a shape reading a preference outranks, which is how a `pt` choice drops `e` from `conjunctions_ambiguous` (decisions/vocabulary-model, 2026-10-27 #753, items 1, 2 and 7; built 2026-10-29, phase 3b). A strict choice, `parser_for(..., strict=True)`, selects by file alone: only the chosen files and native-script vocabulary compile, `mul.toml`'s and `und.toml`'s readings leaving with every other file's, and it too is applied as a delta (decisions/vocabulary-model, 2026-10-30 #753, file-strict; built 2026-10-31, phase 3j). So a bad default is routed around only by choosing the language whose reading is right, which is why #342-class fixes still change shipped vocabulary. The choice is applied as a delta against the default's compile: it adds to the base lexicon what the choice turns on and removes from it what the choice drops, whichever of the default and the caller put a removed entry there, and leaves every other entry, a caller's own included; a pack choosing no language leaves the base's exactly as it was. The choice is one call's: a `Parser` records none, so `parser_for(EN, base=parser_for(PT))` applies `en`'s after `pt`'s and keeps neither language's `do`, where `parser_for(PT, EN)` keeps both (decisions/vocabulary-model, 2026-10-29, chained choices). Pure data means a pack can be audited: a vocabulary pack's whole content is its tag, but what it changes reaches other files too, prefer dropping their readings of its words (a `de` choice drops `ca.toml`'s conjunction `i`), so the audit is the derived surface, `changed_words`, not the pack's file alone.
**How it works.** What is policy for every language (an order constant) is policy, not pack data; packs are lowercase modules exposing uppercase constants; a pack error is wrapped with the pack's code (rule D2). A choice changes the base's readings only by selection, never by a list of removals: a chosen language's `default = false` readings join; on a word a chosen language holds a vocabulary reading of, other languages' vocabulary readings leave, save `mul.toml`'s; and a chosen language's conjunction loses `may_be_initial`, its clash with the initial being a shape reading a preference outranks, which is how a `pt` choice drops `e` from `conjunctions_ambiguous` (decisions/vocabulary-model, 2026-10-27 #753, items 1, 2 and 7; built 2026-10-29, phase 3b). A strict choice, `parser_for(..., strict=True)`, selects by file alone: only the chosen files and native-script vocabulary compile, `mul.toml`'s and `und.toml`'s readings leaving with every other file's (decisions/vocabulary-model, 2026-10-30 #753, file-strict; built 2026-10-31, phase 3j). So a bad default is routed around only by choosing the language whose reading is right, which is why #342-class fixes still change shipped vocabulary. The choice filters the base's shipped readings and the caller's own `add` and `remove` calls apply after it, so a choice never drops a caller's word, and one that clashes with a reading the choice turns on is refused at `parser_for`, naming the edit; a lexicon built from `Lexicon.empty()` holds no shipped reading, so a choice turns none on in it; a pack choosing no language leaves the base's exactly as it was. The choice is recorded on the `Lexicon`, so `parser_for(EN, base=parser_for(PT))` chooses both, as `parser_for(PT, EN)` does (decisions/vocabulary-model, 2026-11-02 #754, the `Lexicon` surface, items 3 and 4; built 2026-11-02, sub-phase 4i-c). Pure data means a pack can be audited: a vocabulary pack's whole content is its tag, but what it changes reaches other files too, prefer dropping their readings of its words (a `de` choice drops `ca.toml`'s conjunction `i`), so the audit is the derived surface, `changed_words`, not the pack's file alone.

**Lives in.** nameparser/locales/ (packs), nameparser/_parser.py (parser_for, the one applier), nameparser/_reading_index.py (the selection, `choose`, and the derived surface, `changed_words`), nameparser/_locale.py (`rules_patch`, a file's compiled `[rules]` as a patch) and tools/vocab/build.py (`RULE_KEYS`, the rules a file may turn on, and `PACK_RULES`, the files the shipped packs read; decisions/vocabulary-model, 2026-10-29 4f). A rule a file turns on is a `Policy` field; a new rule kind is a core change, never code in a pack: a new `Policy` field (with `ADDED_IN`), a row in build.py's `RULE_KEYS` and in `_locale._RULE_VALUES`, and a mechanisms.md entry of its own for the rule kind.
**Lives in.** nameparser/locales/ (packs), nameparser/_parser.py (parser_for, the one applier), nameparser/_lexicon.py (the compile under a recorded choice, `_chosen_pool` and `_built`), nameparser/_reading_index.py (the selection, `choose`, and the derived surface, `changed_words`), nameparser/_locale.py (`rules_patch`, a file's compiled `[rules]` as a patch) and tools/vocab/build.py (`RULE_KEYS`, the rules a file may turn on, and `PACK_RULES`, the files the shipped packs read; decisions/vocabulary-model, 2026-10-29 4f). A rule a file turns on is a `Policy` field; a new rule kind is a core change, never code in a pack: a new `Policy` field (with `ADDED_IN`), a row in build.py's `RULE_KEYS` and in `_locale._RULE_VALUES`, and a mechanisms.md entry of its own for the rule kind.

**Reach for it when.** A language fix wants an if-statement — make it vocabulary or policy in a pack instead.

Expand Down
Loading
Loading