Skip to content

FEAT: Add disemvowel Encoding technique via SearchReplaceConverter - #4449

Open
Roman Lutz (romanlutz) wants to merge 2 commits into
microsoft:mainfrom
romanlutz:gap1-disemvowel
Open

Roman Lutz (romanlutz) wants to merge 2 commits into
microsoft:mainfrom
romanlutz:gap1-disemvowel

Conversation

@romanlutz

@romanlutz Roman Lutz (romanlutz) commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Adds the disemvowel Encoding technique by configuring an existing converter rather than adding a new class (per reuse and compose before adding): SearchReplaceConverter(pattern='[AEIOUaeiou]', replace=''). Keeps the named technique and a focused test.

Disemvowel — remove all vowels. Source: the "disemvoweling" comment-moderation technique (popularized on the Making Light blog).


gap-1 converter series: #4441 #4442 #4443 #4444 #4445 #4446 #4447 #4448 #4449 #4450 #4451 #4452 #4454 #4455 #4456 #4457 #4458

🤖 Generated with Claude Code

Copilot AI balanced review requested due to automatic review settings November 19, 2026 23:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The converter assigns the same parameter identifier to vowel configurations that produce different results because iterable entries are not normalized into individual characters.

1 open issue
What changed in this PR

Adds DisemvowelConverter to PyRIT’s text transformation tools and integrates it into the garak Encoding scenario as an ALL-only technique.

Changes:

  • Implements configurable, case-insensitive vowel removal with component identifiers and lazy public exports.
  • Registers the technique and updates scenario and identifier test data.
  • Adds converter unit tests and a Python documentation example.
File Description
tests/​unit/​scenario/​garak/​test_encoding.py Updates technique counts and ALL-only expectations.
tests/​unit/​registry/​converter_identifier_samples.py Adds custom vowel samples for identifier testing.
tests/​unit/​converter/​test_disemvowel_converter.py Tests conversion, validation, and identifiers.
pyrit/​scenario/​scenarios/​garak/​encoding.py Registers the disemvowel technique and converter.
pyrit/​converter/​disemvowel_converter.py Implements configurable vowel removal.
pyrit/​converter/​__init__.py Exposes the converter through lazy exports.
doc/​code/​converters/​1_text_to_text_converters.py Adds a disemvowel usage example.

Review effort: Balanced

Comment thread pyrit/converter/disemvowel_converter.py Outdated
Raises:
ValueError: If ``vowels`` is empty.
"""
vowel_chars = {char.lower() for char in vowels}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Remove the vowels option. I agree with this comment, and the problem is wider than lists of whole vowel strings. vowels=['ae', 'i'] records the identifier 'aei' but removes only i and I. vowels=[''] passes the empty check and removes nothing. A generator is consumed in init, so DisemvowelConverter(**converter.get_reconstruction_parameters()) raises ValueError. No code in this PR passes vowels, so instead of adding normalization, fix the vowels to a, e, i, o, u; SearchReplaceConverter already covers other character sets. If you keep the option, accept only str, reject the empty string, and build the removal set and the identifier from the same characters.

@richlundeen Richard Lundeen (richlundeen) left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Withdrawn: this earlier suggestion to drop the named converter in favor of configuring an existing one no longer applies. Named converters are welcome; see the current review.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full source review at 5787ebb. The named DisemvowelConverter is welcome. The comments below are about how it is built, its option surface, and the docs and tests that go with it. A reply in the existing vowels thread covers the identifier problem.

Comment thread pyrit/converter/disemvowel_converter.py Outdated
from pyrit.models import ComponentIdentifier, PromptDataType


class DisemvowelConverter(Converter):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Build DisemvowelConverter on SearchReplaceConverter. Keep the named class, but do not reimplement the filter: this class writes its own character loop, input check and identifier, and SearchReplaceConverter already does all three. With the vowels option removed (see the other thread), a subclass whose init calls super().init(pattern='[aeiouAEIOU]', replace='') gives the same output for every Unicode character, including Turkish dotted and dotless I and accented text, and still reconstructs with no parameters. Remove the custom convert_async, _build_identifier and the empty-set check, so later fixes to SearchReplaceConverter also reach this class, and remove the DisemvowelConverter.vowels entry in tests/unit/registry/converter_identifier_samples.py. Do not use re.IGNORECASE: it also deletes U+0130 and U+0131. Also shorten the class docstring to a summary, the 'Hello World' -> 'Hll Wrld' example, and one line that y is kept.

print("ROT13:", await ROT13Converter().convert_async(prompt=prompt)) # type: ignore
print("Base64:", await Base64Converter().convert_async(prompt=prompt)) # type: ignore
print("Base2048:", await Base2048Converter().convert_async(prompt=prompt)) # type: ignore
print("Disemvowel:", await DisemvowelConverter().convert_async(prompt=prompt)) # type: ignore

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Update the notebook pair. This PR adds the DisemvowelConverter import and example to the .py file only, so 1_text_to_text_converters.ipynb, which the docs site shows, has no example and the pair is out of sync. Add the same import and print line, with the output, to the .ipynb, or regenerate it with jupytext --execute. Also re-run the modality table cell in 0_converters.ipynb, because converters.instructions.md requires new converters in that table (it currently also misses BifidConverter).

LeetSpeak = ("leet_speak", {"default"})
AsciiSmuggler = ("ascii_smuggler", set[str]())
Bifid = ("bifid", set[str]())
Disemvowel = ("disemvowel", set[str]())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] Update the garak scanner docs for the new technique. doc/scanner/garak.py and doc/scanner/garak.ipynb still say "18 encodings" and the list ends at Bifid, so users do not see disemvowel. Change the count to 19 and add Disemvowel to the list in both files, keeping the .py and .ipynb in sync. Also add Bifid and Disemvowel to the ALL-only list in the comment above this enum ("(Braille, Ecoji, Zalgo, AsciiSmuggler)").

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] This still applies at 8f51435. The current count is now 20, not the previously suggested 19: ColumnarTransposition has also merged. Both scanner documentation files still say 18 and omit ColumnarTransposition and Disemvowel. The ALL-only comment also omits Bifid, ColumnarTransposition, and Disemvowel.

all_names = {t.value for t in EncodingTechnique.get_all_techniques()}
assert default_names < all_names
assert len(all_names) == 18
assert len(all_names) == 19

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] Add a test that links each technique to a converter variant. If the disemvowel row in _converter_variants is deleted or misspelled, all tests still pass and a disemvowel run builds no attacks. Add one test that each non-aggregate EncodingTechnique has at least one variant row and each row names a real technique. This also covers the other ALL-only techniques.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] The new focused test still does not cover this wiring: it builds a separate SearchReplaceConverter. Removing the actual disemvowel variant only in memory leaves all 68 targeted tests passing. Please exercise the selected Disemvowel technique and its converted raw/decode prompts. One correction to the earlier wording: if that row is missing, selected-only initialization raises IncompatibleTechniqueError in Scenario.initialize_async; an ALL run can still silently omit the technique while running the others.

from pyrit.converter import ConverterResult, DisemvowelConverter


async def test_disemvowel_converter_basic():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] Parametrize the default-conversion tests. Nine tests build DisemvowelConverter() and differ only in input and expected text. Replace them with one parametrized test. Also add match= to both pytest.raises(ValueError) calls, so an unrelated ValueError does not pass, and make the empty-vowels test synchronous because it awaits nothing.

@romanlutz

Copy link
Copy Markdown
Contributor Author

Reworked per your suggestion: dropped DisemvowelConverter and the disemvowel technique is now SearchReplaceConverter(pattern='[AEIOUaeiou]', replace='') in _converter_variants(), keeping the named technique + a focused test. Thanks!

@romanlutz Roman Lutz (romanlutz) changed the title FEAT: Add DisemvowelConverter (disemvowel converter) FEAT: Add disemvowel Encoding technique via SearchReplaceConverter Nov 20, 2026
Per review (reuse and compose before adding): keep the disemvowel technique but implement it with SearchReplaceConverter(pattern='[AEIOUaeiou]', replace='') instead of a dedicated DisemvowelConverter class.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed. No blocking concerns remain. This review records source coverage; merging is handled separately.

[P3] inline:4638732233: Supported F4 remains open. doc/scanner/garak.py:139-141 and garak.ipynb:306-308 still state 18 encodings and end at Bifid. encoding.py:115-133 and the independently rerun actual enum establish 20, including ColumnarTransposition and Disemvowel. encoding.py:112-113's exhaustive ALL-only comment also omits Bifid, ColumnarTransposition, and Disemvowel. The earlier requested 19 is stale; update both scanner formats to 20 and update or simplify the comment. The PR extends this stale catalog to another technique. Documentation/catalog discoverability polish is P3, not a current executable workflow defect.

[P3] inline:4638732238: Supported F5 remains open. test_disemvowel_technique.py:9 builds a separate converter, while test_encoding.py:254-267 checks only enum membership. Reused both exact-head receipts showing 68 targeted passes normally and 68 after process-local removal of the production variant. Independently reran missing-row initialization: selected-only selection raises IncompatibleTechniqueError at scenario.py:1750; ALL still initializes 105 attacks with none for disemvowel. Independently reran actual-row construction: the present row correctly builds five raw/decode attacks. Add a public selected-technique behavior test, not necessarily a private table assertion. This is small regression-sensitivity coverage for currently working configuration-only code, not risky changed control flow or a present runtime failure; retain P3.

[P3] inline:4645599151: Supported useful extension of F4/root inline:4638732233, not a new independent defect. It correctly updates the earlier suggested count from 19 to 20 after ColumnarTransposition merged. Both scanner formats still say 18 and omit both additions; the ALL-only comment omits the three named techniques. Same unresolved P3 docs/catalog correction, retained under both collected IDs without inventing a local finding.

[P3] inline:4645599500: Supported useful extension of F5/root inline:4638732238. It evaluates the new focused test with the supplied independently verified 68-pass missing-row mutation and correctly qualifies the old outcome. This assessor independently reran the actual-row and missing-row probes: the present configuration works; hypothetical removal raises for selected-only but ALL succeeds with 105 other attacks and no disemvowel. The public selected-technique regression-test request remains a P3 improvement. This is an evidence-based extension, not duplicate agreement or a claim that the current row is absent.

[P3] local:F4: Supported, unfixed scanner-catalog cause, identical to inline:4638732233 and its posted extension inline:4645599151. doc/scanner/garak.py:139-141 and garak.ipynb:306-308 still state 18 encodings and end at Bifid; encoding.py:115-133 has 20 concrete techniques, including ColumnarTransposition and Disemvowel, independently confirmed by the prior offline actual-row probe. The ALL-only comment at encoding.py:112-113 omits Bifid, ColumnarTransposition, and Disemvowel. Update the synchronized scanner docs to 20 and update or simplify the comment. The earlier requested 19 is stale. This PR extends the pre-existing catalog omission to a new technique. It is P3 documentation/discoverability polish without a present executable failure. The reply is evidence of reporting, not of repair.

[P3] local:F5: Supported, unfixed scenario-wiring test-sensitivity cause, identical to inline:4638732238 and posted extension inline:4645599500. test_disemvowel_technique.py:9 constructs its own SearchReplaceConverter; test_encoding.py:254-267 checks enum membership rather than the production row. Exact-head full-review and independent-verifier receipts both recorded 68 targeted passes before and after a process-local removal of that row. This assessor independently reran the actual-row probe (five correct raw/decode attacks) and missing-row probe (selected-only IncompatibleTechniqueError at scenario.py:1750; ALL still initializes 105 attacks without disemvowel). A public selected-technique behavior test would detect the hypothetical regression. The current row works, and no risky changed control flow lacks coverage, so this is a P3 regression-sensitivity improvement rather than a current product failure. Posting the conditional correction did not fix the test gap.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants