Skip to content

FEAT: Add FullwidthConverter - #4185

Merged
Roman Lutz (romanlutz) merged 4 commits into
microsoft:mainfrom
parinith-web:feat/fullwidth-converter
Nov 11, 2026
Merged

Roman Lutz (romanlutz) merged 4 commits into
microsoft:mainfrom
parinith-web:feat/fullwidth-converter

Conversation

@parinith-web

Copy link
Copy Markdown
Contributor

Description

Adds FullwidthConverter, a text-to-text converter that maps printable ASCII to the Unicode fullwidth forms (U+FF01–U+FF5E), e.g. Hello! -> Hello!. The output normalizes back to the original under NFKC, so it is a useful probe for filters or classifiers that match exact characters without normalizing input first.

  • New WordLevelConverter subclass in pyrit/converter/fullwidth_converter.py; spaces between words and non-ASCII characters are unchanged, and word selection strategies are supported.
  • Registered in pyrit.converter lazy exports.
  • Unit tests, including an NFKC round trip over all printable ASCII.
  • Added an example to doc/code/converters/1_text_to_text_converters.py. The generated .ipynb files were not re-executed because that notebook includes LLM-based sections that need live endpoints.

Copilot AI balanced review requested due to automatic review settings November 11, 2026 03:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The paired notebook is missing the matching import and example cell, leaving the documentation sources out of sync.

1 open issue
What changed in this PR

Adds FullwidthConverter to PyRIT’s text transformation tools for probing filters that do not normalize Unicode input.

Changes:

  • Maps printable ASCII characters to fullwidth forms with word-selection support.
  • Registers the converter in the public lazy exports and adds a Python documentation example.
  • Adds tests for conversion, ASCII NFKC round trips, word selection, input validation, and stable identifiers.
File Description
tests/​unit/​converter/​test_fullwidth_converter.py Tests conversion behavior and converter contracts.
pyrit/​converter/​fullwidth_converter.py Implements the word-level fullwidth transformation.
pyrit/​converter/​__init__.py Exposes FullwidthConverter through lazy imports.
doc/​code/​converters/​1_text_to_text_converters.py Adds a fullwidth conversion example.

Review effort: Balanced

The table in 0_converters.ipynb is generated at runtime from the registered
converters, so only the stored notebook output was stale. Regenerated the
output cell; no source changes needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The fullwidth converter change has no active review issues requiring further action.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Converting already-fullwidth text is a no-op (code points above U+007E
pass through), which matters when converters are stacked, and astral
code points such as emoji are preserved intact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@romanlutz
Roman Lutz (romanlutz) added this pull request to the merge queue Nov 11, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 11, 2026
@romanlutz
Roman Lutz (romanlutz) added this pull request to the merge queue Nov 11, 2026
Merged via the queue into microsoft:main with commit 714b247 Nov 11, 2026
74 of 75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants