Repository navigation
Record the choice of languages on the Lexicon (#754, 4i-c) - #992
Merged
Merged
Conversation
Lexicon gains _languages/_strict (read as languages/strict, with Parser.languages/strict reading through); a choice filters the base's readings and the caller's log applies after it. parser_for unions a base's choice with its packs', joins the packs' fragments under the caller's lexicon, and refuses a clash naming the edit. _apply_choice and its delta retire. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…754, 4i-c) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ck's mask beat a shipped one (#754, 4i-c review) A chained call choosing no language now keeps base's choice, its strictness included, where the first draft raised on parser_for(locales.RU, base=strict_parser). A shipped mask the caller left unchanged yields to a pack's, as shipped readings do. The kind-set branch of _edits_toward went with the retired delta. The RFC's 3b and 3j passages are put in the past tense. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Omitted (UNSET) keeps base's recorded strictness; strict=False on a strict base raises again, as decided; strict=True on a plain base with a choice makes the union strict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… 4i-c) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…load 4i-b pickles (#754, 4i-c review) parser_for moves base's masks by the choice and lays the packs' over them, as before 4i-c; a failed join or choice compile names the first pack whose fragment fails it (rules/D2), or the languages and the packs that chose them; _languages and _strict carry _ADDED_IN, so a 4i-b pickle loads choosing nothing (fixture written on db7e8fc). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fallback (#754, 4i-c) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…yields to a pack's (#754, 4i-c) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r the translation's branches (#754, 4i-c) The translation of a 2.x agnomen offered .add(w, kind='agnomen') before .add(w, kind='family'), which add() refuses; name readings now come first. The retired choice delta was the other path through the translation's property and kind branches, so the refusal table gains rows reaching them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sub-phase 4i-c of #754: the choice of languages is recorded on the
Lexicon, so a choice composes with a caller's readings instead of being applied as a delta.What changes
Lexicon.languagesandLexicon.strictare read-only, andParser.languagesandParser.strictread through to them. Onlyparser_forsets them publicly. Underneath they are the constructor parameters_languagesand_strict, beside_baseand_edits, sodataclasses.replaceand pickle carry them. They are validated asparser_forvalidates a choice.parser_for(EN, base=parser_for(PT))reads asparser_for(PT, EN):Maria do Carmogives familydo Carmo, andJohn Smith DOgives suffixDO. Under 3b's delta these gave familyCarmoand familyDO.doandvanas particles keeps both underen. Under strictvi, a base that addsdrkeeps it. The delta dropped both.parser_for, naming the edit. For example,locales.CSwith a caller'sstcredential raises:while applying the languages chosen, cs: the lexicon's .add('st', kind='credential') clashes with a reading the choice turns on: ... .remove('st', kind='generation'). The offeredremoveruns, because it applies after the choice.|refuses two different choices or strictnesses, and the error namesparser_for.parser_for'sstricthas three states (bool | _Unset = UNSET), following thesegmenterprecedent:strict=Falseon a strict base: raisesValueError, and the error offers both spellings.strict=Trueon a plain base with a choice: makes the combined choice strict.Lexicon.empty()turns no reading on._parser._apply_choice,_choice_deltaand_choice_readings, along with_lexicon._MARKER_BASESand_edits_toward'shints. The sweeptest_a_choice_written_as_readings_gives_the_deltas_setsis rewritten astest_a_choice_moves_only_the_words_the_callers_log_leaves_alone, over the same 161 tags and three lexicons.parser_for'sraise type(exc)(msg)wrap failed on_lexicon._MaskValueError, whose constructor takes three arguments. The wrap now raises a plainTypeErrororValueError, and a test covers both types.Review round (code review and design-docs review)
Fixed:
strict=Falseon a strict base return a strict parser without saying so. TheUNSETdefault, decided on that review, fixes both, and the explicit-Falserow fails oncf3f442b.en/pt; thelocales.rstwording on an empty base.kind-set branch of_edits_towardwas dead once the delta retired, so it is removed, with its test.Recorded as an accepted limit: a mask the caller deleted comes back when a strict choice is chained, because undoing the base's mask moves cannot tell the caller's own deletions from the choice's. The 4i-c decisions entry carries this limit with its example.
The new rows for the two fixes fail on the pre-fix commit.
Choices made inside the decided design are recorded in the new decisions entry (2026-11-02 #754, sub-phase 4i-c). The main one is the case masks. A mask is no reading, and a choice still moves masks as 3b's delta did (
csgivesjudrthe maskJUDr).parser_formoves them in the field: it undoes the base's choice's moves, then applies the union's, so a chained choice moves the masks as the union does.Docs
_localedocstrings; and a release-log bullet under Unreleased.Measured (over
db7e8fc4, Python 3.11.17)tests/v2/test_snapshot.pyis unchanged.uv run python tools/differential/cldr_agreement.pyreports every view matching its record, the per-pack CLDR and Wikidata native views included, with JA measured with thejaextra installed. No pack moves, because no shipped fragment holds a reading or a mask.tools/perf/call_count.py --against db7e8fc4gives 311.48 on both, and the serial benchmark passes.uv run pytestwith and without namedivider,uv run pytest -n0 -m serial,uv run --python 3.15 pytest,uv run mypy,uv run ruff check,sphinx-build -b doctest docs(586 tests, 0 failures) andpython -m doctest README.rst.Part of #754.
🤖 Generated with Claude Code