You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Invisible bidi control characters survive parsing and pollute output #266
Text copied from web pages, PDFs, or Word documents frequently carries invisible Unicode bidirectional control characters — most commonly the right-to-left mark U+200F around RTL names. The parser doesn't strip them, so they end up embedded in the parsed attributes:
>>>fromnameparserimportHumanName>>>hn=HumanName("\u200fمحمد بن سلمان\u200f")
>>>hn.first'\u200fمحمد'>>>hn.first=="محمد"False
(The \u200f escapes are written out here because the character is invisible; real-world input contains the raw character.)
Because the characters are invisible, the corruption is silent: values print normally but fail equality comparisons, lookups, and deduplication downstream. Constants lookups can also miss (a title or prefix with a mark glued to it no longer matches its set entry).
Suggested fix: strip bidi control characters during pre-processing, exactly as emoji are already handled (#58, re_emoji in nameparser/config/regexes.py). The relevant characters: U+200E (LRM), U+200F (RLM), U+061C (ALM), U+202A–U+202E (embedding/override), U+2066–U+2069 (isolates).
Activity
docs(rules): topic-review amendments from the issue-filing session (p…