Skip to content

FEAT: Add MathematicalAlphanumericConverter (mathematical_alphanumeric converter) - #4451

Open
Roman Lutz (romanlutz) wants to merge 3 commits into
microsoft:mainfrom
romanlutz:gap1-mathalnum
Open

Roman Lutz (romanlutz) wants to merge 3 commits into
microsoft:mainfrom
romanlutz:gap1-mathalnum

Conversation

@romanlutz

@romanlutz Roman Lutz (romanlutz) commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Adds this converter as part of the gap-1 converter series (splitting the esoteric cipher / encoder set into per-converter and per-base PRs). Includes a unit test and garak Encoding-scenario registration; CI-validated locally.

Sibling PRs will be cross-linked once the series is open.

🤖 Generated with Claude Code

gap-1 converter series — splitting the esoteric cipher / encoder set into reviewable PRs (open each independently; they touch the same four registration files, so later ones rebase as earlier ones merge):

…c converter)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 19, 2026 23:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The paired notebook is missing the new import and demonstration and needs its cell and rendered output synchronized with the Python documentation example.

1 open issue
What changed in this PR

Adds MathematicalAlphanumericConverter to PyRIT’s text-conversion toolkit and registers it with the Garak Encoding scenario.

Changes:

  • Supports 13 Unicode styles, including reserved-code-point exceptions and word selection.
  • Exposes the converter through lazy exports and the Encoding scenario’s all selection, without changing defaults.
  • Adds mapping and identifier tests plus a Python documentation example.
File Description
tests/​unit/​scenario/​garak/​test_encoding.py Updates the technique count and non-default selection expectations.
tests/​unit/​registry/​converter_identifier_samples.py Adds style samples for identifier contract checks.
tests/​unit/​converter/​test_mathematical_alphanumeric_converter.py Tests mappings, validation, word selection, and identifiers.
pyrit/​scenario/​scenarios/​garak/​encoding.py Registers the new encoding technique and converter variant.
pyrit/​converter/​mathematical_alphanumeric_converter.py Implements styled Unicode conversion.
pyrit/​converter/​__init__.py Adds the public lazy export and typing import.
doc/​code/​converters/​1_text_to_text_converters.py Adds a converter usage example.

Review effort: Balanced

print("ROT13:", await ROT13Converter().convert_async(prompt=prompt)) # type: ignore
print("Base64:", await Base64Converter().convert_async(prompt=prompt)) # type: ignore
print("Base2048:", await Base2048Converter().convert_async(prompt=prompt)) # type: ignore
print("Mathematical alphanumeric:", await MathematicalAlphanumericConverter().convert_async(prompt=prompt)) # type: ignore

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update the saved modality table too.

.github/instructions/converters.instructions.md:112 also requires the modality table to include new converters. The saved table in doc/code/converters/0_converters.ipynb does not list MathematicalAlphanumericConverter, so readers cannot find it there. Refresh that saved table as well as this paired example. The existing Python code discovers the converter; no manual table row is needed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on this. Handling the reserved "holes" in the Mathematical Alphanumeric block through _STYLE_EXCEPTIONS, and testing that no style emits an unassigned code point, is exactly the right approach. I read the diff but did not run the code or the tests, so these are review notes, not test results.

I spot-checked the base code points for all 13 styles and the exception tables (italic h; script B E F H I L M R e g o; fraktur C H I R Z; double-struck C H N P Q R Z) against my understanding of the Unicode chart, and they look right.

  1. Typing of style. style: str is validated at runtime, but a Literal[...] type (as Base32Converter in #4452 does for variant) would let type checkers and IDEs catch typos. The tuple of valid styles could still be derived from _STYLE_BASES.

  2. Normalization note. The docstring says the output can slip past filters that match on exact ASCII. It might be worth adding that NFKC normalization maps these characters back to plain ASCII, so a target that normalizes its input will undo the transform. That tells users what the converter actually tests.

  3. Line length. In tests/unit/scenario/garak/test_encoding.py the changed assertion assert all_names - default_names == {... "mathematical_alphanumeric"} looks like it comes to roughly 129 characters. If the repo's formatter limit is 120, ruff format or the lint step will want it wrapped.

  4. Scope of the scenario change. The technique is added to the Garak EncodingTechnique set as ALL-only, which bumps the technique count from 18 to 19 in the test. That matches how other non-Garak techniques such as bifid are handled, so I think it is fine, but it may help to mention in the PR description that this one has no Garak counterpart.

  5. Characters not covered. Digits for styles with no digit run, and the dotless ı / ȷ and Greek math letters, are left unchanged. The docstring already explains the digit case, so this is only a note.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the semantic test gap below and the existing paired-notebook synchronization concern. No incorrect production mapping was found; the new finding concerns regression coverage for the handwritten Unicode ranges.


for char in result.output_text:
# unicodedata.name raises ValueError for unassigned code points.
assert unicodedata.name(char)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Check represented letters as well as assigned code points

Please also assert that NFKC-normalizing the full converted alphabet returns alphabet. The current assertion accepts incorrect represented letters: shifting the sans-serif uppercase base by one makes AZ normalize to Ba, yet all 28 new converter tests still pass. The current mapping passes this invariant; adding it here protects the meaning of every mapped letter and digit while retaining the existing check for reserved holes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants