diff --git a/docs/design/decisions/O6-two-family-names.md b/docs/design/decisions/O6-two-family-names.md index 8f91f7562..3c90dd533 100644 --- a/docs/design/decisions/O6-two-family-names.md +++ b/docs/design/decisions/O6-two-family-names.md @@ -9,6 +9,7 @@ - The family's words are counted as P1's fold counts them, a connective join and a hyphen counting as their words, so a family already holding two keeps what it has; the word taken must count as one by the same count and be no initial written alone — 2026-10-24 #754. - A name its script (W4) reads in an order other than the declared `name_order` is left alone; where the two agree it is read like any other — 2026-10-24 #754. - Under a family-first order the first two name words are the family, the remainder laid out as the positions after a family — 2026-10-24 #754. +- Under a family-first order a particle chain after the family is cut where P1's family-first fold cuts its own run, its particles and the one name word they attach to taken, so `Pérez de la Vega Juan` reads family 'Pérez de la Vega', given 'Juan'; without the switch, or under the default order, the chain stays whole — 2026-11-03 #1024. - The switch runs before `middle_as_family` and is not refused beside it: given-first, the two together read as `middle_as_family` alone — 2026-10-24 #754. - The switch reports nothing — 2026-10-24 #754. - The switch reports no alternative either, so the reading it did not take is not surfaced: the caller declared the convention — 2026-10-27 #752 (Derek). @@ -163,3 +164,4 @@ Open: hyphenated surnames block the switch (`Andrés Sánchez-Cid Torres` keeps | ORCID, country SK | 12,962 | 94.84% → 96.40% | 347 | 144 | Every broken record, in all three, has two or more words in the given box, read as a given name and a family name: no population-wide Czech or Slovak given-name list is published to veto the word, the register's frequencies having been withdrawn after 2016's; ČSÚ's top 100 newborn given names a year (CC BY 4.0) is a list, and the open lead. The circular ceiling such a list could reach, and the sources looked for, are in the vocabulary-model entry. +- 2026-11-03 #1024 — UNDER A FAMILY-FIRST ORDER, A PARTICLE CHAIN AFTER THE FAMILY IS CUT WHERE P1'S FAMILY-FIRST FOLD CUTS ITS RUN. rules/O6's Except said a particle chain (P2) is never split, so under FAMILY_FIRST `Pérez de la Vega Juan` reached O6 as one unit, `de la Vega Juan`, which counts two name words, and O6 declined: family 'Pérez', given 'de la Vega Juan', under both family-first orders, the switch on or off. The caller has declared two things the chain contradicts, an order putting the given name after the family (decisions/P1's 2026-08-17 #395 entry reads a declared family-first order as evidence of where the surname ends) and a family of two names; and a chain running on through `Juan` makes the given name part of the surname P2 says a particle is written as part of, which is `Vega`. So where the first unit after the family begins with a particle and runs past the one name word its particles attach to, O6 cuts it as `_fold_reach` cuts P1's run, takes the particles and that word as the second family name, and lays out what is left by the order: `Pérez de la Vega Juan Carlos` gives given 'Juan', middle 'Carlos' under FAMILY_FIRST and middle 'Juan', given 'Carlos' under FAMILY_FIRST_GIVEN_LAST, as `García Márquez Juan Carlos` reads. The rule's own example `Juan de la Vega Rodríguez` (family 'Juan' under FAMILY_FIRST, where it illustrated the decline) is this shape, and now reads family 'Juan de la Vega', given 'Rodríguez'; the decline's example is `Pérez de la Vega`, where taking the chain would leave no given name. Unchanged: the switch off (whether the chain's reach should follow a family-first order by itself, which would move `Nguyen Van Thanh Long`, is #1024's open question 1, not decided here), the default order, a family comma, and a chain whose first unit is a conjunction join (`Pérez de la Vega y Santos Juan`, the unit `de la Vega y Santos` counting two words, so O6 still declines). With the switch on, a Vietnamese name with `Van` after the family now gives up a word to the family as any name of three or more words does under the switch (`Nguyen Van Thanh Long` → family 'Nguyen Van Thanh', given 'Long'), the switch declaring two family names. No ambiguity is reported, as for every reading of the switch. A lone particle a family-first order left as the family takes the cut chain after it as it takes any word (rules/O6's Accepted), so `Van der Merwe Jan` reads family 'Van der Merwe', surname2 'der Merwe', where it read family 'Van', given 'der Merwe Jan': the family field right, one surname read as two (the review; corpus examples `van den Berg Cornelia …`). Under the default order nothing is cut, held by test_cases' `two_family_names_cuts_no_chain_under_the_default_order`, whose recorded negative control without the order guard reads `du Ménil Thi Mai` family 'Thi Mai'; 29 corpus names move under that mutant. Measured 2026-11-03 by the issue's recipe on the cached `orcid-researchers` corpus (ES, MX, CO, PE, CL, VE, AR, at most 20,000 records a country with both boxes, 135,735 records): of the 1,577 whose family box holds a particle after its first word, the family box is read right under `Policy(name_order=FAMILY_FIRST, two_family_names=True)` in 2 on 39839b91 and in 1,273 on the branch, and of the 1,296 in the issue's exact shape (one word, a particle run, one word) in 1 and 1,272; by country, of the 1,577, ES 264/383, MX 164/221, CO 59/63, PE 152/199, CL 113/142, VE 406/433, AR 115/136. Recompute: the issue's recipe, run against each tree with `nameparser.__file__` asserted; the comparator is the base tree, so 2 and 1 are the same parser's reading before the change. diff --git a/docs/design/rules/O6-two-family-names.md b/docs/design/rules/O6-two-family-names.md index b5b3d6e47..eccbc54f5 100644 --- a/docs/design/rules/O6-two-family-names.md +++ b/docs/design/rules/O6-two-family-names.md @@ -16,6 +16,15 @@ "García Márquez Gabriel" family-first+two_family_names → given="Gabriel" "García Márquez Ana María" family-first-given-last+two_family_names → given="María" ``` +- **Under a family-first order, a particle chain after the family is cut where P1's family-first fold cuts its own run**: its particles and the one name word they attach to are the word taken, and what follows is laid out by the order, as after any word the rule takes. + ``` + "Pérez de la Vega Juan" family-first+two_family_names → family="Pérez de la Vega" + "Pérez de la Vega Juan" family-first+two_family_names → given="Juan" + "Pérez de la Vega Juan Carlos" family-first+two_family_names → middle="Carlos" + "Pérez de la Vega Juan Carlos" family-first-given-last+two_family_names → given="Carlos" + "Juan de la Vega Rodríguez" family-first+two_family_names → family="Juan de la Vega" + "Pérez de la Vega Juan" family-first → given="de la Vega Juan" · boundary + ``` - **A name word is counted as P1's fold counts one**, so a particle run counts with the word it attaches to: "de la Vega" is one word and takes the word before it. ``` "Juan Pérez de la Vega" two_family_names → family="Pérez de la Vega" @@ -59,7 +68,7 @@ ``` ", Ana García López" two_family_names → family="" ``` -- **What is taken must be one name word a family name could be**: a part counting as two words (a hyphenated given name, a bound given-name pair, P5) stays where it is, and so do an initial written alone, which writes a given name, and a mark or number with no letter in it. A particle chain (P2) is never split, so where taking it would leave no given name the rule declines. A listed given name stays where it is too, under either order, and a list written without accents matches the word written with them: the caller's vocabulary names it a word only a given name is, so the name holds one family name. A listed word behind a particle is a surname unit, taken as before. +- **What is taken must be one name word a family name could be**: a part counting as two words (a hyphenated given name, a bound given-name pair, P5) stays where it is, and so do an initial written alone, which writes a given name, and a mark or number with no letter in it. A particle chain (P2) is split only where the family-first bullet cuts it and otherwise moves whole, and where taking it would leave no given name the rule declines. A listed given name stays where it is too, under either order, and a list written without accents matches the word written with them: the caller's vocabulary names it a word only a given name is, so the name holds one family name. A listed word behind a particle is a surname unit, taken as before. ``` "José Luis Pérez" two_family_names+es_given_names → family="Pérez" "Jose Luis Perez" two_family_names+es_given_names → middle="Luis" @@ -70,7 +79,8 @@ "Ana M. García" two_family_names → family="García" "Ana María García - López" two_family_names → family="López" "Ana María 3 López" two_family_names → family="López" - "Juan de la Vega Rodríguez" family-first+two_family_names → family="Juan" + "Pérez de la Vega" family-first+two_family_names → family="Pérez" + "Pérez de la Vega y Santos Juan" family-first+two_family_names → given="de la Vega y Santos Juan" ``` - **An order a name's script set is not a declared one** (W4), and no script it keys writes two family names as separate words, so a name its script reads in an order other than the declared `name_order` is left alone. ``` @@ -104,9 +114,10 @@ ``` "Petrov Ivan Sergeevich" family-first+two_family_names → family="Petrov Ivan" ``` -- A lone particle the declared order left as the family counts as one word, R2 reading it as an ordinary name word, so the word after it joins. +- A lone particle the declared order left as the family counts as one word, R2 reading it as an ordinary name word, so the word after it joins, and so does a particle chain after it, cut as the family-first bullet cuts one: one surname written with two particles reads as two. ``` "van Gogh Ana María" family-first+two_family_names → family="van Gogh" + "Van der Merwe Jan" family-first+two_family_names → surname2="der Merwe" ``` - Where a name's script order and the declared one agree, nothing tells the two apart, so under a declared family-first order a spaced Han or Hangul name is read like any other. ``` diff --git a/docs/design/rules/P2-particles-join-forward.md b/docs/design/rules/P2-particles-join-forward.md index 560e6c4f4..8c7ba8611 100644 --- a/docs/design/rules/P2-particles-join-forward.md +++ b/docs/design/rules/P2-particles-join-forward.md @@ -30,6 +30,10 @@ ``` "de la Cruz Juan Carlos" family-first → family="de la Cruz" ``` +- **Where two-family-name handling takes the chain after a family-first family, it takes the same part** the fold would, leaving to the order the given name written after it (O6). + ``` + "Pérez de la Vega Juan" family-first+two_family_names → given="Juan" + ``` **Because:** a particle is written as part of the surname it precedes, and a title stands outside the name entirely. diff --git a/docs/release_log.rst b/docs/release_log.rst index 60867ba54..af49192f5 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -38,6 +38,7 @@ Release Log **Behavior Changes** - **Fix particles joined by a connective reading the connective as the family's base.** ``parse("Fritz Freiherr und von Bar")`` gives family_base ``Freiherr und von Bar`` and initials ``F. F. v. B.``, where 2.0.0 through 2.8.0 gave family_base ``und``; a connective ending the family is still its base (``Juan de y`` gives ``y``), and one joining particles before a name word stays in it (``Alois von und zu Liechtenstein`` gives ``und Liechtenstein``). :issue:`939` + - **Fix two_family_names under a family-first order taking the given name into a particle chain.** ``Parser(policy=Policy(name_order=FAMILY_FIRST, two_family_names=True)).parse("Pérez de la Vega Juan")`` gives family ``Pérez de la Vega``, given ``Juan``, where 2.6.0 through 2.8.0 gave family ``Pérez``, given ``de la Vega Juan``; the chain's particles and the one name word they attach to are the second family name, and without the switch the chain stays whole. :issue:`1024` - **Read the Spanish professor titles Profesor and Profesora as titles.** ``parse("Juan Pérez, Profesor")`` gives title ``Profesor``, given ``Juan``, family ``Pérez``, where 1.4.0 through 2.8.0 gave given ``Profesor``, family ``Juan Pérez``, and ``Profesora Ana López`` reads the same way; the Polish and Romanian spelling is the same word. :issue:`804` - **Render a name in the order it was read in, keeping its written separators.** ``str(parse("김민준"))`` gives ``'김민준'``, where 2.1.0 through 2.8.0 gave ``'민준 김'``; a name read family-first renders family-first, and a comma before a suffix stays (``John Smith, Jr.``, ``Smith, Jr.``), while a family comma between name words never does (``Smith, John`` gives ``'John Smith'``). A spec you pass to ``render()`` renders as written, so 2.x's default spec keeps given-first output, and a name pickled by 2.x renders as it did. ``initials()`` gives nothing for a word in Han, Hangul or kana. The new ``ParsedName.order`` records the order; ``render()``'s spec now defaults to ``None``. See :doc:`migrate`. :issue:`754` - **Fix a credential written in Title case before a name word taking the name with it.** ``parse("Badrul Akmal Hisham Md Yusoff")`` gives middle ``Akmal Hisham Md``, family ``Yusoff``, as 1.4.0 through 2.3.0 did, where 2.4.0 through 2.8.0 gave family ``Hisham``, suffix ``Md Yusoff``; ``MD``, ``M.D.`` and the part after a family comma (``Smith, John Md Jones``) still start the credential's run. :issue:`1019` diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index b50768d51..52a92d8c9 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -287,6 +287,15 @@ def _take_second_family_name(state: ParseState, # The rest in the units assign read them as, so a particle chain # (P2) or a bound given-name pair (P5) moves whole or not at all. units = _units(tokens, others) + # rules/O6: "a particle chain after the family is cut where P1's + # family-first fold cuts its own run" -- under a family-first order, + # the given name the order declares after the chain left to the + # order (decisions/O6, #1024); under the default order the chain + # ends the name and stays whole + if state.order[0] is not Role.GIVEN and units: + cut = _fold_reach(tokens, units[0]) + if cut < len(units[0]): + units[:1] = [units[0][:cut], *_units(tokens, units[0][cut:])] if len(units) < 2: return take = units[-1] if state.order[0] is Role.GIVEN else units[0] diff --git a/tests/v2/cases.py b/tests/v2/cases.py index b9b890e7c..b7f550448 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -534,13 +534,31 @@ def _check_cjk_shape_purity(self) -> None: classification="feat(#754)", notes="P1's family-first fold leaves family 'de Mesnil', given " "'Jean', middle 'Paul'; O6 then takes one unit more"), - Case("two_family_names_moves_a_particle_chain_whole_or_not_at_all", + Case("two_family_names_cuts_a_family_first_chain_as_the_fold_does", "Juan de la Vega Rodríguez", - {"given": "de la Vega Rodríguez", "family": "Juan"}, + {"given": "Rodríguez", "family": "Juan de la Vega"}, policy=Policy(name_order=FAMILY_FIRST, two_family_names=True), - classification="parity", - notes="the given name is one P2 chain; taking it whole would " - "leave no given name, so O6 declines"), + classification="feat(#1024)", + notes="the chain after the family is cut where P1's family-first " + "fold cuts its run, the particles and Vega taken, the " + "given name the order declares after them left to it; " + "2.8.0 read given 'de la Vega Rodríguez', family 'Juan'"), + Case("two_family_names_leaves_a_chain_that_is_the_whole_given_part", + "Pérez de la Vega", + {"given": "de la Vega", "family": "Pérez"}, + policy=Policy(name_order=FAMILY_FIRST, two_family_names=True), + classification="feat(#754)", + notes="taking the chain would leave no given name, so O6 " + "declines"), + Case("two_family_names_cuts_no_chain_under_the_default_order", + "du Ménil Thi Mai", + {"given": "du", "middle": "Ménil Thi", "family": "Mai"}, + ambiguities=("particle-or-given",), + policy=_TWO, classification="feat(#754)", + notes="#1024's cut is the family-first order's: under the " + "default order a chain opening the name is not cut. " + "Negative control, recorded 2026-11-03: without the " + "order guard this reads family 'Thi Mai'"), Case("two_family_names_keeps_the_particle_p6_attached", "Beethoven Ludwig Maria van", {"given": "Maria", "family": "van Beethoven Ludwig"}, diff --git a/tests/v2/rules_budget.json b/tests/v2/rules_budget.json index a31199544..cdb9b83e8 100644 --- a/tests/v2/rules_budget.json +++ b/tests/v2/rules_budget.json @@ -39,10 +39,10 @@ "O3": {"words": 47, "then_words": 18, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "O4": {"words": 91, "then_words": 57, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "O5": {"words": 478, "then_words": 160, "because_sentences": 2, "except": 6, "precedence": 1, "accepted": 0, "accepted_words": 0}, - "O6": {"words": 661, "then_words": 170, "because_sentences": 2, "except": 5, "precedence": 4, "accepted": 8, "accepted_words": 297}, + "O6": {"words": 721, "then_words": 220, "because_sentences": 2, "except": 5, "precedence": 4, "accepted": 8, "accepted_words": 321}, "O7": {"words": 598, "then_words": 175, "because_sentences": 2, "except": 4, "precedence": 3, "accepted": 6, "accepted_words": 220}, "P1": {"words": 415, "then_words": 287, "because_sentences": 2, "except": 0, "precedence": 1, "accepted": 4, "accepted_words": 319}, - "P2": {"words": 315, "then_words": 279, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 3, "accepted_words": 167}, + "P2": {"words": 344, "then_words": 308, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 3, "accepted_words": 167}, "P3": {"words": 639, "then_words": 598, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "P4": {"words": 190, "then_words": 82, "because_sentences": 1, "except": 1, "precedence": 0, "accepted": 0, "accepted_words": 0}, "P5": {"words": 717, "then_words": 399, "because_sentences": 1, "except": 2, "precedence": 3, "accepted": 3, "accepted_words": 192}, diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index db1e69a30..053764baa 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -345,9 +345,10 @@ def test_the_field_sweep_sees_the_claims_it_checks() -> None: # #913's 'Saint Clair Soares' and 'Mr Saint Martin Sterling', each # reporting title-or-name under three orders; 1542 -> 1545 the same # day, #1037's 'Delos Wheeler', reporting particle-or-given under - # three orders. - assert claims == 1545, ( - f"the field sweep checks {claims} claims, recorded as 1545 on " + # three orders; 1545 -> 1546 the same day, #1024's 'du Ménil Thi + # Mai', reporting particle-or-given. + assert claims == 1546, ( + f"the field sweep checks {claims} claims, recorded as 1546 on " f"2026-11-03") diff --git a/tests/v2/test_types.py b/tests/v2/test_types.py index 1a35f9c08..ceb144dfd 100644 --- a/tests/v2/test_types.py +++ b/tests/v2/test_types.py @@ -1331,6 +1331,10 @@ def test_a_patronymic_parsedname_pickled_by_2_8_0_loads_equal() -> None: ("García Márquez Ana María", _FFGL, True, "Márquez"), ("de Mesnil Jean Paul", _FF, False, "Jean"), ("van Gogh Ana María", _FF, False, "Gogh"), + # a chain after the family is cut as P1's fold cuts it (#1024) + ("Juan de la Vega Rodríguez", _FF, False, "de la Vega"), + ("Pérez de la Vega Juan Carlos", _FFGL, False, "de la Vega"), + ("Pérez de la Vega Juan Carlos", _FF, True, "de la Vega"), # P6's particle attaches to the first unit and stays out ("Beethoven Ludwig Maria van", _FF, False, "Ludwig"), ("Beethoven Ludwig Maria van", _FF, True, "Ludwig"), @@ -1355,7 +1359,7 @@ def test_a_patronymic_parsedname_pickled_by_2_8_0_loads_equal() -> None: ("Ana María-José López", _GF, False, ""), ("Ana abdul Rahman López", _GF, False, ""), ("Ana M. García", _GF, False, ""), - ("Juan de la Vega Rodríguez", _FF, False, ""), + ("Pérez de la Vega", _FF, False, ""), ("王 小 明", _GF, False, ""), ] diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 75bb4411e..63e638dfe 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -887,6 +887,10 @@ "Prof. Smith, John, PhD" "Pérez José Luis" "Pérez Luis José" +"Pérez de la Vega" +"Pérez de la Vega Juan" +"Pérez de la Vega Juan Carlos" +"Pérez de la Vega y Santos Juan" "Pérez de la Vega, Juan" "Pérez, Antonio Juan Carlos" "Pérez, Juan Carlos" @@ -1086,6 +1090,7 @@ "VI, AD" "Van Buren, Ed" "Van Johnson" +"Van der Merwe Jan" "Vega, Juan de la" "Vincent van Gogh van Beethoven" "Wang M.Eng." diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 5da89c332..991071bcd 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -3092,6 +3092,10 @@ {"name": "Pérez José Luis", "given": "Pérez", "middle": "José", "family": "Luis", "initials": "P. J. L."} {"name": "Pérez Lucía", "order": "FAMILY_FIRST", "given": "Lucía", "family": "Pérez", "initials": "L. P."} {"name": "Pérez Luis José", "given": "Pérez", "middle": "Luis", "family": "José", "initials": "P. L. J."} +{"name": "Pérez de la Vega", "given": "Pérez", "family": "de la Vega", "initials": "P. V."} +{"name": "Pérez de la Vega Juan", "given": "Pérez", "family": "de la Vega Juan", "initials": "P. V. J."} +{"name": "Pérez de la Vega Juan Carlos", "given": "Pérez", "family": "de la Vega Juan Carlos", "initials": "P. V. J. C."} +{"name": "Pérez de la Vega y Santos Juan", "given": "Pérez", "family": "de la Vega y Santos Juan", "initials": "P. V. S. J."} {"name": "Pérez de la Vega, Juan", "given": "Juan", "family": "Pérez de la Vega", "initials": "J. P. V."} {"name": "Pérez, Antonio Juan Carlos", "given": "Antonio", "middle": "Juan Carlos", "family": "Pérez", "initials": "A. J. C. P."} {"name": "Pérez, Juan Carlos", "given": "Juan", "middle": "Carlos", "family": "Pérez", "initials": "J. C. P."} @@ -3754,6 +3758,7 @@ {"name": "Van Maximilian Carlson", "given": "Van", "middle": "Maximilian", "family": "Carlson", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: particle as family (vocabulary)"], "initials": "V. M. C."} {"name": "Van Nguyen", "given": "Van", "family": "Nguyen", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: particle as family (vocabulary)"], "initials": "V. N."} {"name": "Van Rooten, Luis", "given": "Luis", "family": "Van Rooten", "initials": "L. R.", "capitalized_force": "Luis van Rooten"} +{"name": "Van der Merwe Jan", "given": "Van", "family": "der Merwe Jan", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: particle as family (vocabulary)"], "initials": "V. M. J."} {"name": "Van der Merwe Jan Koos", "order": "FAMILY_FIRST", "given": "der Merwe Jan Koos", "family": "Van", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: particle as family (vocabulary)"], "initials": "d. M. J. K. V.", "capitalized_force": "Van Der Merwe Jan Koos"} {"name": "Van der Merwe, Jan Koos", "given": "Jan", "middle": "Koos", "family": "Van der Merwe", "initials": "J. K. M.", "capitalized_force": "Jan Koos van der Merwe"} {"name": "Van der Walt", "given": "Van", "family": "der Walt", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: particle as family (vocabulary)"], "initials": "V. W."}