Repository navigation
FEAT: Add MathematicalAlphanumericConverter (mathematical_alphanumeric converter) - #4451
Roman Lutz (romanlutz) wants to merge 3 commits into
Conversation
…c converter) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
The paired notebook is missing the new import and demonstration and needs its cell and rendered output synchronized with the Python documentation example.
1 open issue
What changed in this PR
Adds MathematicalAlphanumericConverter to PyRIT’s text-conversion toolkit and registers it with the Garak Encoding scenario.
Changes:
- Supports 13 Unicode styles, including reserved-code-point exceptions and word selection.
- Exposes the converter through lazy exports and the Encoding scenario’s
allselection, without changing defaults. - Adds mapping and identifier tests plus a Python documentation example.
| File | Description |
|---|---|
tests/unit/scenario/garak/test_encoding.py |
Updates the technique count and non-default selection expectations. |
tests/unit/registry/converter_identifier_samples.py |
Adds style samples for identifier contract checks. |
tests/unit/converter/test_mathematical_alphanumeric_converter.py |
Tests mappings, validation, word selection, and identifiers. |
pyrit/scenario/scenarios/garak/encoding.py |
Registers the new encoding technique and converter variant. |
pyrit/converter/mathematical_alphanumeric_converter.py |
Implements styled Unicode conversion. |
pyrit/converter/__init__.py |
Adds the public lazy export and typing import. |
doc/code/converters/1_text_to_text_converters.py |
Adds a converter usage example. |
Review effort: Balanced
| print("ROT13:", await ROT13Converter().convert_async(prompt=prompt)) # type: ignore | ||
| print("Base64:", await Base64Converter().convert_async(prompt=prompt)) # type: ignore | ||
| print("Base2048:", await Base2048Converter().convert_async(prompt=prompt)) # type: ignore | ||
| print("Mathematical alphanumeric:", await MathematicalAlphanumericConverter().convert_async(prompt=prompt)) # type: ignore |
There was a problem hiding this comment.
Update the saved modality table too.
.github/instructions/converters.instructions.md:112 also requires the modality table to include new converters. The saved table in doc/code/converters/0_converters.ipynb does not list MathematicalAlphanumericConverter, so readers cannot find it there. Refresh that saved table as well as this paired example. The existing Python code discovers the converter; no manual table row is needed.
Varun Poojari (Varunpoojari)
left a comment
There was a problem hiding this comment.
Nice work on this. Handling the reserved "holes" in the Mathematical Alphanumeric block through _STYLE_EXCEPTIONS, and testing that no style emits an unassigned code point, is exactly the right approach. I read the diff but did not run the code or the tests, so these are review notes, not test results.
I spot-checked the base code points for all 13 styles and the exception tables (italic h; script B E F H I L M R e g o; fraktur C H I R Z; double-struck C H N P Q R Z) against my understanding of the Unicode chart, and they look right.
-
Typing of
style.style: stris validated at runtime, but aLiteral[...]type (asBase32Converterin #4452 does forvariant) would let type checkers and IDEs catch typos. The tuple of valid styles could still be derived from_STYLE_BASES. -
Normalization note. The docstring says the output can slip past filters that match on exact ASCII. It might be worth adding that NFKC normalization maps these characters back to plain ASCII, so a target that normalizes its input will undo the transform. That tells users what the converter actually tests.
-
Line length. In
tests/unit/scenario/garak/test_encoding.pythe changed assertionassert all_names - default_names == {... "mathematical_alphanumeric"}looks like it comes to roughly 129 characters. If the repo's formatter limit is 120,ruff formator the lint step will want it wrapped. -
Scope of the scenario change. The technique is added to the Garak
EncodingTechniqueset as ALL-only, which bumps the technique count from 18 to 19 in the test. That matches how other non-Garak techniques such asbifidare handled, so I think it is fine, but it may help to mention in the PR description that this one has no Garak counterpart. -
Characters not covered. Digits for styles with no digit run, and the dotless
ı/ȷand Greek math letters, are left unchanged. The docstring already explains the digit case, so this is only a note.
Richard Lundeen (richlundeen)
left a comment
There was a problem hiding this comment.
Please address the semantic test gap below and the existing paired-notebook synchronization concern. No incorrect production mapping was found; the new finding concerns regression coverage for the handwritten Unicode ranges.
|
|
||
| for char in result.output_text: | ||
| # unicodedata.name raises ValueError for unassigned code points. | ||
| assert unicodedata.name(char) |
There was a problem hiding this comment.
[P2] Check represented letters as well as assigned code points
Please also assert that NFKC-normalizing the full converted alphabet returns alphabet. The current assertion accepts incorrect represented letters: shifting the sans-serif uppercase base by one makes AZ normalize to Ba, yet all 28 new converter tests still pass. The current mapping passes this invariant; adding it here protects the meaning of every mapped letter and digit while retaining the existing check for reserved holes.

Adds this converter as part of the gap-1 converter series (splitting the esoteric cipher / encoder set into per-converter and per-base PRs). Includes a unit test and garak Encoding-scenario registration; CI-validated locally.
Sibling PRs will be cross-linked once the series is open.
🤖 Generated with Claude Code
gap-1 converter series — splitting the esoteric cipher / encoder set into reviewable PRs (open each independently; they touch the same four registration files, so later ones rebase as earlier ones merge):