diff --git a/docs/design/decisions/O6-two-family-names.md b/docs/design/decisions/O6-two-family-names.md index 3c90dd533..16f26aab2 100644 --- a/docs/design/decisions/O6-two-family-names.md +++ b/docs/design/decisions/O6-two-family-names.md @@ -6,7 +6,7 @@ - `locales.ES` turns the switch on, carrying the Spanish given-name list beside it, one pack for Spain and Latin America alike, Argentina included; `locales.PT` leaves it off until a Portuguese given-name list exists, decided in chat — 2026-10-30 #753, phase 3g. - `locales.CS` and `locales.SK` turn the switch on with no given-name list, a net gain in every real population measured (CLDR's fictional renderings lose), two given names the recorded cost; `locales.SL` and `locales.HR` leave it off, decided in chat — 2026-10-30 #753, phase 3f amended. - Both family names stay in `family` through 2.x; a second-surname field is the RFC's Phase 4 output model (docs/design/vocabulary-model-rfc.md, "The rules a pack can choose"), and a pack may later switch this rule on — 2026-10-24 #754; landed as the `surname2` view, `family` keeping both, 2026-10-29 #754, sub-phase 4d. -- The family's words are counted as P1's fold counts them, a connective join and a hyphen counting as their words, so a family already holding two keeps what it has; the word taken must count as one by the same count and be no initial written alone — 2026-10-24 #754. +- The family's words are counted as P1's fold counts them, a connective join and a hyphen counting as their words, so a family already holding two keeps what it has; the word taken must count as one, a hyphenated word counting as one there unless it writes given names (each part a listed given name or an initial), and be no initial written alone — 2026-10-24 #754; 2026-11-03 #959 (Derek). - A name its script (W4) reads in an order other than the declared `name_order` is left alone; where the two agree it is read like any other — 2026-10-24 #754. - Under a family-first order the first two name words are the family, the remainder laid out as the positions after a family — 2026-10-24 #754. - Under a family-first order a particle chain after the family is cut where P1's family-first fold cuts its own run, its particles and the one name word they attach to taken, so `Pérez de la Vega Juan` reads family 'Pérez de la Vega', given 'Juan'; without the switch, or under the default order, the chain stays whole — 2026-11-03 #1024. @@ -17,7 +17,7 @@ - The veto stays a clause of this rule though it reads a second signal, the taken word looked up in a list, and the rule is a split candidate — 2026-10-30 #959, decided. - The Spanish list, which `locales.ES` carries as its lexicon fragment since sub-phase 3g (a caller may pass a list of its own), is generated, not per-word vocabulary: INE's 2025 census tables, pooled over every name borne in Spain, cut at a measured rank (1,000 words, 90% given use) by `tools/es_given_names.py`; a change to it is a cutoff change — 2026-10-30 #959. -Open: hyphenated surnames block the switch (`Andrés Sánchez-Cid Torres` keeps `Sánchez-Cid` a middle name), of the 466 Spanish records the issue's held-out per-country list still read wrong, 271 are this shape (measured on the issue, 2026-10-30) — #959. No rule example shows both switches on over a given name a particle completes (`María del Carmen de la Vega`, given `María del Carmen` under O7 alone and family `del Carmen de la Vega` with O6 on too) — #959. Why the Spanish list does less in Latin America, and whether a list per region or another selection rule would serve it better than the one list `locales.ES` carries (the 2026-10-30 #959 entry's Latin America bullet; the 2026-10-30 #753 phase 3g entry measures the pack against the default there) — #959. A name written with ñ dropped (`Begona`, `Inigo`) matches no entry, the key keeping the tilde; whether to fold ñ for data that strips it — #959. +Open: why the Spanish list does less in Latin America, and whether a list per region or another selection rule would serve it better than the one list `locales.ES` carries (the 2026-10-30 #959 entry's Latin America bullet; the 2026-10-30 #753 phase 3g entry measures the pack against the default there) — #959. A name written with ñ dropped (`Begona`, `Inigo`) matches no entry, the key keeping the tilde; whether to fold ñ for data that strips it — #959. ## Log @@ -165,3 +165,5 @@ Open: hyphenated surnames block the switch (`Andrés Sánchez-Cid Torres` keeps Every broken record, in all three, has two or more words in the given box, read as a given name and a family name: no population-wide Czech or Slovak given-name list is published to veto the word, the register's frequencies having been withdrawn after 2016's; ČSÚ's top 100 newborn given names a year (CC BY 4.0) is a list, and the open lead. The circular ceiling such a list could reach, and the sources looked for, are in the vocabulary-model entry. - 2026-11-03 #1024 — UNDER A FAMILY-FIRST ORDER, A PARTICLE CHAIN AFTER THE FAMILY IS CUT WHERE P1'S FAMILY-FIRST FOLD CUTS ITS RUN. rules/O6's Except said a particle chain (P2) is never split, so under FAMILY_FIRST `Pérez de la Vega Juan` reached O6 as one unit, `de la Vega Juan`, which counts two name words, and O6 declined: family 'Pérez', given 'de la Vega Juan', under both family-first orders, the switch on or off. The caller has declared two things the chain contradicts, an order putting the given name after the family (decisions/P1's 2026-08-17 #395 entry reads a declared family-first order as evidence of where the surname ends) and a family of two names; and a chain running on through `Juan` makes the given name part of the surname P2 says a particle is written as part of, which is `Vega`. So where the first unit after the family begins with a particle and runs past the one name word its particles attach to, O6 cuts it as `_fold_reach` cuts P1's run, takes the particles and that word as the second family name, and lays out what is left by the order: `Pérez de la Vega Juan Carlos` gives given 'Juan', middle 'Carlos' under FAMILY_FIRST and middle 'Juan', given 'Carlos' under FAMILY_FIRST_GIVEN_LAST, as `García Márquez Juan Carlos` reads. The rule's own example `Juan de la Vega Rodríguez` (family 'Juan' under FAMILY_FIRST, where it illustrated the decline) is this shape, and now reads family 'Juan de la Vega', given 'Rodríguez'; the decline's example is `Pérez de la Vega`, where taking the chain would leave no given name. Unchanged: the switch off (whether the chain's reach should follow a family-first order by itself, which would move `Nguyen Van Thanh Long`, is #1024's open question 1, not decided here), the default order, a family comma, and a chain whose first unit is a conjunction join (`Pérez de la Vega y Santos Juan`, the unit `de la Vega y Santos` counting two words, so O6 still declines). With the switch on, a Vietnamese name with `Van` after the family now gives up a word to the family as any name of three or more words does under the switch (`Nguyen Van Thanh Long` → family 'Nguyen Van Thanh', given 'Long'), the switch declaring two family names. No ambiguity is reported, as for every reading of the switch. A lone particle a family-first order left as the family takes the cut chain after it as it takes any word (rules/O6's Accepted), so `Van der Merwe Jan` reads family 'Van der Merwe', surname2 'der Merwe', where it read family 'Van', given 'der Merwe Jan': the family field right, one surname read as two (the review; corpus examples `van den Berg Cornelia …`). Under the default order nothing is cut, held by test_cases' `two_family_names_cuts_no_chain_under_the_default_order`, whose recorded negative control without the order guard reads `du Ménil Thi Mai` family 'Thi Mai'; 29 corpus names move under that mutant. Measured 2026-11-03 by the issue's recipe on the cached `orcid-researchers` corpus (ES, MX, CO, PE, CL, VE, AR, at most 20,000 records a country with both boxes, 135,735 records): of the 1,577 whose family box holds a particle after its first word, the family box is read right under `Policy(name_order=FAMILY_FIRST, two_family_names=True)` in 2 on 39839b91 and in 1,273 on the branch, and of the 1,296 in the issue's exact shape (one word, a particle run, one word) in 1 and 1,272; by country, of the 1,577, ES 264/383, MX 164/221, CO 59/63, PE 152/199, CL 113/142, VE 406/433, AR 115/136. Recompute: the issue's recipe, run against each tree with `nameparser.__file__` asserted; the comparator is the base tree, so 2 and 1 are the same parser's reading before the change. +- 2026-11-03 #959 (Derek) — A HYPHENATED WORD COUNTS AS ONE WHEN IT IS THE WORD TAKEN, UNLESS EVERY PART OF IT IS A LISTED GIVEN NAME OR AN INITIAL. SUPERSEDES, in part, the 2026-10-24 #754 entry's count of the word taken, in which a hyphen joined two words without making them one, so `Andrés Sánchez-Cid Torres` kept `Sánchez-Cid` a middle name (the #754 review's reading of `Ana María-José López`, middle 'María-José', which the first draft had read as family 'María-José López'). The issue found this shape behind 271 of the 466 Spanish records the given-name veto still read wrong. Measured 2026-11-03 on the cached `orcid-researchers` corpus, ES, MX, CO, PE, CL, VE and AR at most 20,000 records a country with both boxes (135,735 records), over the 10,794 distinct records holding a hyphen in either box, rendered given first and read right where both the family and the given names match the boxes (`local_agreement.fold`), under `parser_for(locales.ES)`, on 39839b91 against prototypes of three readings: a hyphen counting as one everywhere O6 counts (family and word taken) reads 10,038 right against 10,281, fixing 277 and breaking 520, most of them a family box of two surnames written with a hyphen (`Garcia-Gomez`), which then counts as one and takes a given name; a hyphen counting as one in the word taken alone reads 10,464, fixing 186 (ES 125, MX 24, CL 20, PE 6, CO 5, VE 5, AR 1) and breaking 3 (`Aristide Tanyi-Jong Akem`, `Hilda Ines-Romero Menacho`, `Benjamin Moraga-Arias .`); and the same with a veto on a hyphenated word every part of which is a listed given name, which moves none of these records. Decided: the word-taken count with the veto, the veto keeping the rule's own `Ana María-José López` where the Spanish list is loaded (middle 'María-José'), while without a list the switch takes it (family 'María-José López'), its usual cost (rules/O6's Accepted). Found building it and in its review, and folded into the veto, which reads each part: a part that is an initial counts as a listed given name does, so hyphenated initials (`Ana M.-J. López`) stay as an initial written alone stays, and `M.-José` stays under the list, which moves no ORCID record. A hyphenated bound given-name pair (`Ana Abdul-Rahman López`, middle 'Abdul-Rahman' in 2.8.0) is taken like any hyphenated word, family 'Abdul-Rahman López', where the pair written as two words (P5) stays: a veto on a hyphenated word whose first part is a bound given name was built and backed out the same day, since in this population it cost 2 records whose hyphenated word is a surname (`Paola Valentina Abdul-Hadi Pru`, `Giorgia Anette Abd-El-Kader Espinoza`) and kept none, the hyphen being how those surnames are written; rules/O6's Accepted records the cost. A hyphen in the family still counts its words, so `Javier Olmedo Pérez-Montaut` keeps `Olmedo` a middle name: reading `Pérez-Montaut` as one surname is the 520 above. The built change reads the same 10,464 and moves the same 186 and 3, and with the switch alone (`Policy(two_family_names=True)`, no list) 10,292 to 10,475, the same records. Recompute: for each record of `local_agreement.records(local_corpora.load("orcid-researchers"))` in those countries, both boxes non-empty, at most 20,000 a country, keep those with a hyphen (`_post_rules._HYPHENS`) in either box, distinct by given box, family box and country (two records are duplicates), parse `f"{given} {family}"` with each tree's parser, `nameparser.__file__` asserted, and compare. +- 2026-11-03 #959 — BOTH SWITCHES OVER A GIVEN NAME A PARTICLE COMPLETES GET AN EXAMPLE. The 2026-10-30 entry's Open named the shape no rule example showed: `María del Carmen de la Vega` reads family 'del Carmen de la Vega' with both switches on, O6 taking the chain before O7 can complete the given name with it, and given 'María del Carmen' under O7 alone. rules/O7's precedence bullet against two-family-name handling now says so and carries both rows; no behavior moves. diff --git a/docs/design/rules/O6-two-family-names.md b/docs/design/rules/O6-two-family-names.md index eccbc54f5..0ae462d32 100644 --- a/docs/design/rules/O6-two-family-names.md +++ b/docs/design/rules/O6-two-family-names.md @@ -25,9 +25,11 @@ "Juan de la Vega Rodríguez" family-first+two_family_names → family="Juan de la Vega" "Pérez de la Vega Juan" family-first → given="de la Vega Juan" · boundary ``` -- **A name word is counted as P1's fold counts one**, so a particle run counts with the word it attaches to: "de la Vega" is one word and takes the word before it. +- **A name word is counted as P1's fold counts one**, so a particle run counts with the word it attaches to: "de la Vega" is one word and takes the word before it. A hyphenated word counts as one when it is the word taken, where a hyphen in the family counts its words (Except). ``` "Juan Pérez de la Vega" two_family_names → family="Pérez de la Vega" + "Andrés Sánchez-Cid Torres" two_family_names → family="Sánchez-Cid Torres" + "Torres Sánchez-Cid Andrés" family-first+two_family_names → family="Torres Sánchez-Cid" ``` - **The family keeps the two as units**, the one written second being the second-surname view (R10) and the one written first the first-surname views (R11), whichever of the two the rule took. ``` @@ -68,13 +70,15 @@ ``` ", Ana García López" two_family_names → family="" ``` -- **What is taken must be one name word a family name could be**: a part counting as two words (a hyphenated given name, a bound given-name pair, P5) stays where it is, and so do an initial written alone, which writes a given name, and a mark or number with no letter in it. A particle chain (P2) is split only where the family-first bullet cuts it and otherwise moves whole, and where taking it would leave no given name the rule declines. A listed given name stays where it is too, under either order, and a list written without accents matches the word written with them: the caller's vocabulary names it a word only a given name is, so the name holds one family name. A listed word behind a particle is a surname unit, taken as before. +- **What is taken must be one name word a family name could be**: a bound given-name pair (P5) written as two words stays where it is, and so do an initial written alone, which writes a given name, and a mark or number with no letter in it. A particle chain (P2) is split only where the family-first bullet cuts it and otherwise moves whole, and where taking it would leave no given name the rule declines. A listed given name stays where it is too, under either order, and so does a hyphenated word every part of which is one or an initial, and a list written without accents matches the word written with them: the caller's vocabulary names it a word only a given name is, so the name holds one family name. A listed word behind a particle is a surname unit, taken as before. ``` "José Luis Pérez" two_family_names+es_given_names → family="Pérez" "Jose Luis Perez" two_family_names+es_given_names → middle="Luis" "Pérez José Luis" family-first+two_family_names+es_given_names → family="Pérez" "Pérez Luis José" family-first-given-last+two_family_names+es_given_names → family="Pérez" - "Ana María-José López" two_family_names → family="López" + "Ana María-José López" two_family_names+es_given_names → family="López" + "Ana M.-J. López" two_family_names → family="López" + "Ana M.-José López" two_family_names+es_given_names → family="López" "Ana abdul Rahman López" two_family_names → family="López" "Ana M. García" two_family_names → family="García" "Ana María García - López" two_family_names → family="López" @@ -109,6 +113,8 @@ - A name with one family name behind two given names loses its second given name to the family, which is the shape the switch cannot tell from the one it is for: most Argentine and English names of three words or more read wrong under it, and so do Portuguese and Brazilian names whose writer gave the last family name alone as the family. That is why it is off by default and a caller turns it on only for data written this way. ``` "Juan Carlos Pérez" two_family_names → family="Carlos Pérez" + "Ana María-José López" two_family_names → family="María-José López" + "Ana Abdul-Rahman López" two_family_names → family="Abdul-Rahman López" ``` - A patronymic is a name word like any other here, a patronymic rule on or off, so where no rotation fired it can join the family. ``` diff --git a/docs/design/rules/O7-two-given-names.md b/docs/design/rules/O7-two-given-names.md index 4e624333b..5eb2aa3cc 100644 --- a/docs/design/rules/O7-two-given-names.md +++ b/docs/design/rules/O7-two-given-names.md @@ -68,7 +68,7 @@ **Precedence:** -- **Against two-family-name handling** ("Gabriel García Márquez" with both switches on): that rule reads first, since the caller who turns both on says the family holds two names, and this rule takes from the middles it leaves. A name of three words then keeps one given name, and a name of four whose third word that rule takes reads two and two. A listed given name that rule leaves where it is (O6) is a middle, which this rule takes where it stands beside the given name, so a name of three words whose middle is listed reads two given names and one family name. +- **Against two-family-name handling** ("Gabriel García Márquez" with both switches on): that rule reads first, since the caller who turns both on says the family holds two names, and this rule takes from the middles it leaves. A name of three words then keeps one given name, and a name of four whose third word that rule takes reads two and two. A listed given name that rule leaves where it is (O6) is a middle, which this rule takes where it stands beside the given name, so a name of three words whose middle is listed reads two given names and one family name. A given name a particle completes is a particle chain that rule takes into the family first, so it is the family's with both on, where this rule alone reads it as the given name. ``` "Gabriel García Márquez" two_family_names+two_given_names → given="Gabriel" "José Luis Pérez" two_family_names+two_given_names+es_given_names → given="José Luis" @@ -76,6 +76,8 @@ "Ana María Luis Pérez" two_family_names+two_given_names+es_given_names → middle="Luis" "María José García López" two_family_names+two_given_names → given="María José" "María José García López" two_family_names+two_given_names → family="García López" + "María del Carmen de la Vega" two_family_names+two_given_names → family="del Carmen de la Vega" + "María del Carmen de la Vega" two_given_names → given="María del Carmen" ``` - **Against compound-family handling** ("Hassan Mohamad Ali" with both switches on): this rule reads first, as a bound given-name pair (P5) joins before the fold, and O3 then folds whatever middles are left. ``` diff --git a/docs/release_log.rst b/docs/release_log.rst index af49192f5..7b031b3b2 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -37,6 +37,7 @@ Release Log **Behavior Changes** + - **Fix two_family_names leaving a hyphenated surname out of the family.** ``Parser(policy=Policy(two_family_names=True)).parse("Andrés Sánchez-Cid Torres")`` gives family ``Sánchez-Cid Torres``, where 2.6.0 through 2.8.0 gave middle ``Sánchez-Cid``, family ``Torres``; a hyphenated given name is taken too unless every part of it is a listed given name, as under ``locales.ES`` (``Ana María-José López`` keeps middle ``María-José``), and hyphenated initials stay. :issue:`959` - **Fix particles joined by a connective reading the connective as the family's base.** ``parse("Fritz Freiherr und von Bar")`` gives family_base ``Freiherr und von Bar`` and initials ``F. F. v. B.``, where 2.0.0 through 2.8.0 gave family_base ``und``; a connective ending the family is still its base (``Juan de y`` gives ``y``), and one joining particles before a name word stays in it (``Alois von und zu Liechtenstein`` gives ``und Liechtenstein``). :issue:`939` - **Fix two_family_names under a family-first order taking the given name into a particle chain.** ``Parser(policy=Policy(name_order=FAMILY_FIRST, two_family_names=True)).parse("Pérez de la Vega Juan")`` gives family ``Pérez de la Vega``, given ``Juan``, where 2.6.0 through 2.8.0 gave family ``Pérez``, given ``de la Vega Juan``; the chain's particles and the one name word they attach to are the second family name, and without the switch the chain stays whole. :issue:`1024` - **Read the Spanish professor titles Profesor and Profesora as titles.** ``parse("Juan Pérez, Profesor")`` gives title ``Profesor``, given ``Juan``, family ``Pérez``, where 1.4.0 through 2.8.0 gave given ``Profesor``, family ``Juan Pérez``, and ``Profesora Ana López`` reads the same way; the Polish and Romanian spelling is the same word. :issue:`804` diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index 52a92d8c9..40c6a7883 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -42,7 +42,7 @@ import bisect import re -from nameparser._lexicon import _normalize, _run_addresses_by_given +from nameparser._lexicon import Lexicon, _normalize, _run_addresses_by_given from nameparser._pipeline._assign import ( _name_alt, _name_pieces, _name_positions, ) @@ -56,7 +56,7 @@ WorkToken, _NEVER_FLIPPED, comma_bucket, copy_with, ) from nameparser._pipeline._vocab import ( - delimiter_cores, has_patronymic_ending, is_listed_given_name, + delimiter_cores, has_patronymic_ending, is_initial, is_listed_given_name, is_patronymic_marker, suffix_reading, unit_ends, ) from nameparser._policy import PatronymicRule @@ -252,21 +252,34 @@ def _fold_reach(tokens: list[WorkToken], name_idx: list[int]) -> int: _HYPHENS = re.compile("[-\u2010\u2011\u2012\u2013\u2212\ufe63\uff0d]") -def _family_word_count(tokens: list[WorkToken], family: list[int]) -> int: +def _family_word_count(tokens: list[WorkToken], family: list[int], + hyphen_joins: bool = False) -> int: """The family's name words as rules/O6 counts them: a particle run counts with the word it attaches to, a connective counts as nothing and a hyphen joins words without making them one, so "de la Vega" is one word and "Ortega y Gasset" and "García-López" two. A family of nothing but particles counts each of them, R2 reading - them as ordinary name words.""" - words = sum(len([part for part in _HYPHENS.split(tokens[i].text) - if part]) + them as ordinary name words. `hyphen_joins` counts a hyphenated word + as one, as the word O6 takes is counted (#959).""" + words = sum(1 if hyphen_joins + else len([part for part in _HYPHENS.split(tokens[i].text) + if part]) for i in family if "particle" not in tokens[i].tags and "conjunction" not in tokens[i].tags) return words or len(family) +def _hyphenated_given(text: str, lexicon: Lexicon) -> bool: + """A hyphenated word that writes given names, which the taken word's + count (a hyphenated word counting as one) would otherwise let through + (#959): every part of it a listed given name or an initial + ("María-José" under the Spanish list, "M.-J.", "M.-José").""" + parts = [part for part in _HYPHENS.split(text) if part] + return len(parts) > 1 and all(is_initial(part) or is_listed_given_name(part, lexicon) + for part in parts) + + def _take_second_family_name(state: ParseState, tokens: list[WorkToken]) -> None: # rules/O6: "the family name takes the name word beside it, so @@ -299,12 +312,13 @@ def _take_second_family_name(state: ParseState, if len(units) < 2: return take = units[-1] if state.order[0] is Role.GIVEN else units[0] - # What is taken must be ONE name word, counted as the family is, so - # a hyphenated given name or a bound given-name pair (P5) stays put; - # and a word a family name could be: not an initial, which writes a - # given name (one written alone; "J.R." is taken), nor a lone dash - # or other mark with no letter in it. - if _family_word_count(tokens, take) != 1: + # What is taken must be ONE name word, so a bound given-name pair + # (P5) stays put; rules/O6: "A hyphenated word counts as one when it + # is the word taken" (decisions/O6, #959), where the family's own + # hyphen counts two; and a word a family name could be: not an + # initial, which writes a given name (one written alone; "J.R." is + # taken), nor a lone dash or other mark with no letter in it. + if _family_word_count(tokens, take, hyphen_joins=True) != 1: return if len(take) == 1 and "initial" in tokens[take[0]].tags: return @@ -316,8 +330,10 @@ def _take_second_family_name(state: ParseState, # either order. Only a unit of one token is asked: a word # behind a particle ("de Miguel") is a unit of two, a surname taken # as before. History: decisions/O6 (#959). - if len(take) == 1 and is_listed_given_name(tokens[take[0]].text, - state.lexicon): + # A hyphenated word writing given names stays too (#959). + if len(take) == 1 and ( + is_listed_given_name(tokens[take[0]].text, state.lexicon) + or _hyphenated_given(tokens[take[0]].text, state.lexicon)): return if state.order[0] is Role.GIVEN: # given, middles..., family: the middle unit standing right diff --git a/nameparser/_policy.py b/nameparser/_policy.py index c65b24af5..98b54d2b0 100644 --- a/nameparser/_policy.py +++ b/nameparser/_policy.py @@ -699,9 +699,12 @@ class Policy: #: García-López", "Ana María de la Vega Rodríguez"), titles and #: suffixes count as no name word, and a name of two name words is #: unchanged. The word taken must be one name word a family name - #: could be, so an initial written alone ("Ana M. García"), a - #: hyphenated given name ("Ana María-José López") and a bound - #: given-name pair stay where they are. Under a family-first + #: could be, so an initial written alone ("Ana M. García"), hyphenated + #: initials ("Ana M.-J. López") and a bound given-name pair stay where + #: they are; a hyphenated word counts as one when + #: taken ("Andrés Sánchez-Cid Torres" gives family ``Sánchez-Cid + #: Torres``), so a hyphenated given name is taken unless every part + #: of it is a listed given name or an initial (``Lexicon.given_names``). Under a family-first #: name_order the FIRST two name words are the family; a name its #: script reads in an order other than name_order (a spaced Han or #: Hangul name under the default) is left alone, though under a diff --git a/tests/v2/cases.py b/tests/v2/cases.py index b7f550448..b2e9c4d82 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -581,13 +581,27 @@ def _check_cjk_shape_purity(self) -> None: {"given": "Ana", "middle": "M.", "family": "García"}, policy=_TWO, classification="parity", notes="#754 review: an initial writes a given name"), - Case("two_family_names_leaves_a_hyphenated_given_name_before_the_family", + Case("two_family_names_takes_a_hyphenated_word_as_one", + "Andrés Sánchez-Cid Torres", + {"given": "Andrés", "family": "Sánchez-Cid Torres"}, + policy=_TWO, classification="feat(#959)", + notes="a hyphenated word counts as one when it is the word " + "taken; 2.8.0 kept 'Sánchez-Cid' a middle name, its two " + "words counted as the family's are"), + Case("two_family_names_takes_a_hyphenated_given_name_without_a_list", "Ana María-José López", - {"given": "Ana", "middle": "María-José", "family": "López"}, + {"given": "Ana", "family": "María-José López"}, + policy=_TWO, classification="feat(#959)", + notes="the switch's cost (O6's Accepted): with no given-name " + "list nothing says María-José is a given name; the " + "#754 review's reading, middle 'María-José', is what " + "the Spanish list's veto gives"), + Case("two_family_names_leaves_hyphenated_initials_before_the_family", + "Ana M.-J. López", + {"given": "Ana", "middle": "M.-J.", "family": "López"}, policy=_TWO, classification="parity", - notes="#754 review: the hyphen's two words count against the " - "taken word as they count in the family; the first draft " - "read family 'María-José López'"), + notes="hyphenated initials write given names, as an initial " + "written alone does (#959)"), Case("two_family_names_leaves_a_bound_given_pair_before_the_family", "Ana abdul Rahman López", {"given": "Ana", "middle": "abdul Rahman", "family": "López"}, diff --git a/tests/v2/pipeline/test_post_rules.py b/tests/v2/pipeline/test_post_rules.py index cedfa4120..77e27ada0 100644 --- a/tests/v2/pipeline/test_post_rules.py +++ b/tests/v2/pipeline/test_post_rules.py @@ -1360,8 +1360,12 @@ def _roles(text: str, policy: Policy, lexicon: Lexicon) -> tuple[str, ...]: ("José", "Luis Pérez", ""), ("José", "Luis Pérez", "")), ("Ana Martín López", GIVEN_FIRST, ("Ana", "", "Martín López"), ("Ana", "", "Martín López")), + # a hyphenated word taken counts as one, and the veto reads each + # of its parts (#959): every part listed, it stays ("Ana María-Luis López", GIVEN_FIRST, - ("Ana", "María-Luis", "López"), ("Ana", "María-Luis", "López")), + ("Ana", "", "María-Luis López"), ("Ana", "María-Luis", "López")), + ("Ana María-Torres López", GIVEN_FIRST, + ("Ana", "", "María-Torres López"), ("Ana", "", "María-Torres López")), ("José Luis Pérez-García", GIVEN_FIRST, ("José", "Luis", "Pérez-García"), ("José", "Luis", "Pérez-García")), # a listed word behind a particle is a surname unit, taken as @@ -1387,7 +1391,7 @@ def test_the_veto_counts_its_rows() -> None: moved = [r for r in _VETO_EFFECT if r[2] != r[3]] assert {r[1] for r in moved} == {GIVEN_FIRST, FAMILY_FIRST, FAMILY_FIRST_GIVEN_LAST} - assert len(moved) == 11 and len(_VETO_EFFECT) - len(moved) == 7 + assert len(moved) == 12 and len(_VETO_EFFECT) - len(moved) == 7 def test_the_veto_leaves_the_word_to_two_given_names() -> None: diff --git a/tests/v2/rules_budget.json b/tests/v2/rules_budget.json index cdb9b83e8..d58cb51d6 100644 --- a/tests/v2/rules_budget.json +++ b/tests/v2/rules_budget.json @@ -39,8 +39,8 @@ "O3": {"words": 47, "then_words": 18, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "O4": {"words": 91, "then_words": 57, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "O5": {"words": 478, "then_words": 160, "because_sentences": 2, "except": 6, "precedence": 1, "accepted": 0, "accepted_words": 0}, - "O6": {"words": 721, "then_words": 220, "because_sentences": 2, "except": 5, "precedence": 4, "accepted": 8, "accepted_words": 321}, - "O7": {"words": 598, "then_words": 175, "because_sentences": 2, "except": 4, "precedence": 3, "accepted": 6, "accepted_words": 220}, + "O6": {"words": 752, "then_words": 242, "because_sentences": 2, "except": 5, "precedence": 4, "accepted": 8, "accepted_words": 321}, + "O7": {"words": 633, "then_words": 175, "because_sentences": 2, "except": 4, "precedence": 3, "accepted": 6, "accepted_words": 220}, "P1": {"words": 415, "then_words": 287, "because_sentences": 2, "except": 0, "precedence": 1, "accepted": 4, "accepted_words": 319}, "P2": {"words": 344, "then_words": 308, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 3, "accepted_words": 167}, "P3": {"words": 639, "then_words": 598, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, diff --git a/tests/v2/test_types.py b/tests/v2/test_types.py index ceb144dfd..3859559f8 100644 --- a/tests/v2/test_types.py +++ b/tests/v2/test_types.py @@ -1356,7 +1356,7 @@ def test_a_patronymic_parsedname_pickled_by_2_8_0_loads_equal() -> None: ("Ana María de la Vega Rodríguez", _GF, False, ""), # O6's other declines ("Ana García", _GF, False, ""), - ("Ana María-José López", _GF, False, ""), + ("Ana M.-J. López", _GF, False, ""), ("Ana abdul Rahman López", _GF, False, ""), ("Ana M. García", _GF, False, ""), ("Pérez de la Vega", _FF, False, ""), diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 63e638dfe..d5a8b356b 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -57,12 +57,15 @@ "Aliyev, oglu" "Alves, Pedro Fils" "Alves, Pedro Neto" +"Ana Abdul-Rahman López" "Ana Fo" "Ana García" "Ana García (Anita) López" "Ana J.R. García" "Ana Jorge Flores" "Ana M. García" +"Ana M.-J. López" +"Ana M.-José López" "Ana Martín López" "Ana María 3 López" "Ana María García - López" @@ -85,6 +88,7 @@ "Andrew Perkins (MBA)" "Andrew Perkins (PhD)" "Andrew Perkins «MBA»" +"Andrés Sánchez-Cid Torres" "Anh Do" "Anh Van Do" "Anh van Do" @@ -1087,6 +1091,7 @@ "The Mahdi Muhammad Ahmad" "The Rev. Deacon John Smith" "The Right Hon. the President of the Queen's Bench Division" +"Torres Sánchez-Cid Andrés" "VI, AD" "Van Buren, Ed" "Van Johnson" diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 991071bcd..6d182436f 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -206,12 +206,15 @@ {"name": "Amélie Beaury-Saurel", "given": "Amélie", "family": "Beaury-Saurel", "initials": "A. B."} {"name": "Amīn as-Sulṭān ʿAlī Aṣġar Ḫān", "given": "Amīn", "middle": "as-Sulṭān ʿAlī Aṣġar", "family": "Ḫān", "initials": "A. a. ʿ. A. Ḫ.", "capitalized_force": "Amīn As-Sulṭān ʿAlī Aṣġar Ḫān"} {"name": "An tOll. An Dr. Brigid Aoibhe Sophia de Bhulbh", "given": "An", "middle": "tOll. An Dr. Brigid Aoibhe Sophia", "family": "de Bhulbh", "initials": "A. t. A. D. B. A. S. B.", "capitalized_force": "An Toll. An Dr. Brigid Aoibhe Sophia de Bhulbh"} +{"name": "Ana Abdul-Rahman López", "given": "Ana", "middle": "Abdul-Rahman", "family": "López", "initials": "A. A. L."} {"name": "Ana Fo", "given": "Ana", "family": "Fo", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: agnomen as suffix (vocabulary)"], "initials": "A. F."} {"name": "Ana García", "given": "Ana", "family": "García", "initials": "A. G."} {"name": "Ana García (Anita) López", "given": "Ana", "middle": "García", "family": "López", "nickname": "Anita", "initials": "A. G. L."} {"name": "Ana J.R. García", "given": "Ana", "middle": "J.R.", "family": "García", "initials": "A. J. G."} {"name": "Ana Jorge Flores", "given": "Ana", "middle": "Jorge", "family": "Flores", "initials": "A. J. F."} {"name": "Ana M. García", "given": "Ana", "middle": "M.", "family": "García", "initials": "A. M. G."} +{"name": "Ana M.-J. López", "given": "Ana", "middle": "M.-J.", "family": "López", "initials": "A. M. L."} +{"name": "Ana M.-José López", "given": "Ana", "middle": "M.-José", "family": "López", "initials": "A. M. L."} {"name": "Ana Marija Vera Kovač", "given": "Ana", "middle": "Marija Vera", "family": "Kovač", "initials": "A. M. V. K."} {"name": "Ana Martín López", "given": "Ana", "middle": "Martín", "family": "López", "initials": "A. M. L."} {"name": "Ana María 3 López", "given": "Ana", "middle": "María 3", "family": "López", "initials": "A. M. 3. L."} @@ -256,6 +259,7 @@ {"name": "Andrew Perkins, Jr., Col. (Ret)", "title": "Col.", "given": "Andrew", "family": "Perkins", "suffix": "Jr., Ret", "initials": "A. P.", "generation": "Jr.", "credentials": "Ret"} {"name": "Andrews, M.D.", "family": "Andrews", "suffix": "M.D.", "initials": "A."} {"name": "Andrés David Padilla Mejía", "given": "Andrés", "middle": "David Padilla", "family": "Mejía", "initials": "A. D. P. M."} +{"name": "Andrés Sánchez-Cid Torres", "given": "Andrés", "middle": "Sánchez-Cid", "family": "Torres", "initials": "A. S. T."} {"name": "Ane Miren Urrestaratzu", "given": "Ane", "middle": "Miren", "family": "Urrestaratzu", "initials": "A. M. U."} {"name": "Angiviller, Charles Claude de La Billarderie d'", "given": "Charles", "middle": "Claude de La Billarderie", "family": "d' Angiviller", "initials": "C. C. B. A.", "capitalized_force": "Charles Claude de la Billarderie d' Angiviller"} {"name": "Angoulême, Louis Antoine d'Artois d'", "given": "Louis", "middle": "Antoine d'Artois", "family": "d' Angoulême", "initials": "L. A. d. A.", "capitalized_force": "Louis Antoine D'Artois d' Angoulême"} @@ -3706,6 +3710,7 @@ {"name": "Tomás Iglesias Pérez", "given": "Tomás", "middle": "Iglesias", "family": "Pérez", "initials": "T. I. P."} {"name": "Tony Miranda", "given": "Tony", "family": "Miranda", "initials": "T. M."} {"name": "Torres Ana", "order": "FAMILY_FIRST", "given": "Ana", "family": "Torres", "initials": "A. T."} +{"name": "Torres Sánchez-Cid Andrés", "given": "Torres", "middle": "Sánchez-Cid", "family": "Andrés", "initials": "T. S. A."} {"name": "Torres, Ana", "given": "Ana", "family": "Torres", "initials": "A. T."} {"name": "Tounens, Antoine de", "given": "Antoine", "family": "de Tounens", "initials": "A. T."} {"name": "Traberth, Julie", "given": "Julie", "family": "Traberth", "initials": "J. T."}