Skip to content

FEAT: Add RegionalIndicatorConverter (regional_indicator converter) - #4450

Open
Roman Lutz (romanlutz) wants to merge 3 commits into
microsoft:mainfrom
romanlutz:gap1-regional
Open

Roman Lutz (romanlutz) wants to merge 3 commits into
microsoft:mainfrom
romanlutz:gap1-regional

Conversation

@romanlutz

@romanlutz Roman Lutz (romanlutz) commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Adds this converter as part of the gap-1 converter series (splitting the esoteric cipher / encoder set into per-converter and per-base PRs). Includes a unit test and garak Encoding-scenario registration; CI-validated locally.

Sibling PRs will be cross-linked once the series is open.

🤖 Generated with Claude Code

gap-1 converter series — splitting the esoteric cipher / encoder set into reviewable PRs (open each independently; they touch the same four registration files, so later ones rebase as earlier ones merge):

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 19, 2026 23:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The paired notebook is missing the new import and example, leaving the documentation representations out of sync with repository requirements.

1 open issue
What changed in this PR

Adds RegionalIndicatorConverter to PyRIT’s text transformations and makes it available in the garak Encoding scenario.

Changes:

  • Converts ASCII letters to regional indicator symbols, separating adjacent tiles to prevent flag formation.
  • Exports the converter and registers an ALL-only encoding technique.
  • Adds conversion tests, updates scenario expectations, and adds a Python documentation example.
File Description
tests/​unit/​scenario/​garak/​test_encoding.py Updates technique counts and ALL-only expectations.
tests/​unit/​converter/​test_regional_indicator_converter.py Tests conversion, separators, passthrough, and supported types.
pyrit/​scenario/​scenarios/​garak/​encoding.py Registers the regional indicator technique and converter variant.
pyrit/​converter/​regional_indicator_converter.py Implements regional indicator text conversion.
pyrit/​converter/​__init__.py Adds lazy export and typing support.
doc/​code/​converters/​1_text_to_text_converters.py Adds a converter usage example.

Review effort: Balanced

print("ROT13:", await ROT13Converter().convert_async(prompt=prompt)) # type: ignore
print("Base64:", await Base64Converter().convert_async(prompt=prompt)) # type: ignore
print("Base2048:", await Base2048Converter().convert_async(prompt=prompt)) # type: ignore
print("Regional indicator:", await RegionalIndicatorConverter().convert_async(prompt=prompt)) # type: ignore

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still applies at the current head: the paired notebook has neither the import nor the example. Also update the saved modality table in 0_converters.ipynb, which omits RegionalIndicatorConverter; the converter instructions require that reference update. Its source already discovers converters dynamically, so no hard-coded row is needed.

Verified on commit 085f5aa95f50f863e6dd2935f666c6013b5aa575.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two P2 issues remain in the new conversion and scenario-selection paths; details are inline. The existing notebook synchronization concern also remains open.

if previous_was_indicator:
result.append(self._SEPARATOR)
result.append(chr(self._REGIONAL_INDICATOR_A + ord(char.lower()) - ord("a")))
previous_was_indicator = True

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Separate generated tiles from existing regional indicators

With prompt='\U0001f1faS' or prompt='U\U0001f1f8', this returns \U0001f1fa\U0001f1f8 without U+200C, so a newly converted letter forms a flag with an existing indicator. The passthrough branch resets the state and never separates either mixed boundary. Base separation on adjacent emitted indicators when at least one was converted, and add cases for both directions while preserving existing flags.

assert len(all_names) == 19
# The niche/lossy schemes stay ALL-only.
assert all_names - default_names == {"braille", "ecoji", "zalgo", "ascii_smuggler", "bifid"}
assert all_names - default_names == {"braille", "ecoji", "zalgo", "ascii_smuggler", "bifid", "regional_indicator"}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Test that the new technique builds its converter attacks

These assertions check only the enum: removing the regional_indicator row from Encoding._converter_variants() still lets all 42 tests in the two changed files pass, while selecting EncodingTechnique.RegionalIndicator builds no attacks. Add one isolated scenario-selection test that checks the raw/decode variants and converter application, so a missing or misnamed registration is detected.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants