Skip to content

FEAT: Add ubbi_dubbi Encoding technique via SearchReplaceConverter - #4448

Open
Roman Lutz (romanlutz) wants to merge 2 commits into
microsoft:mainfrom
romanlutz:gap1-ubbidubbi
Open

Roman Lutz (romanlutz) wants to merge 2 commits into
microsoft:mainfrom
romanlutz:gap1-ubbidubbi

Conversation

@romanlutz

@romanlutz Roman Lutz (romanlutz) commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Adds the ubbi_dubbi Encoding technique via an existing converter (reuse before adding): SearchReplaceConverter(pattern=r'([aeiouAEIOU]+)', replace=r'ub\1') (prefixes each vowel run with ub, source case preserved). Keeps the named technique and a focused test.

Ubbi Dubbi — a language game. Source: popularized by PBS's Zoom (1972).


gap-1 converter series: #4441 #4442 #4443 #4444 #4445 #4446 #4447 #4448 #4449 #4450 #4451 #4452 #4454 #4455 #4456 #4457 #4458

🤖 Generated with Claude Code

Copilot AI balanced review requested due to automatic review settings November 19, 2026 23:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The converter’s regex can stall the event loop through quadratic backtracking on punctuation-heavy inputs and leaves newline-containing tokens unencoded.

1 open issue
What changed in this PR

Adds a configurable Ubbi Dubbi encoder to PyRIT’s converter library and makes it available in the Garak Encoding scenario.

Changes:

  • Implements vowel-group encoding with configurable vowels and word selection.
  • Exports the converter and registers ubbi_dubbi without changing the scenario’s default technique set.
  • Adds unit coverage, identifier samples, and a Python documentation example.
File Description
tests/​unit/​scenario/​garak/​test_encoding.py Updates technique counts and ALL-only expectations.
tests/​unit/​registry/​converter_identifier_samples.py Adds vowel configurations for identifier checks.
tests/​unit/​converter/​test_ubbi_dubbi_converter.py Tests encoding, capitalization, punctuation, and validation.
pyrit/​scenario/​scenarios/​garak/​encoding.py Registers the Ubbi Dubbi technique and converter variant.
pyrit/​converter/​ubbi_dubbi_converter.py Implements the configurable word-level converter.
pyrit/​converter/​__init__.py Adds lazy export and typing support.
doc/​code/​converters/​1_text_to_text_converters.py Adds an import and usage example.

Review effort: Balanced

Comment thread pyrit/converter/ubbi_dubbi_converter.py Outdated
Comment on lines +10 to +13
# Splits a whitespace token into (leading non-letters, alphabetic-ish core, trailing non-letters).
# The core is non-greedy so leading/trailing punctuation is peeled off while interior characters
# (e.g. the apostrophe in "don't") stay inside the core and are passed through untouched.
_TOKEN_PATTERN = re.compile(r"^([^A-Za-z]*)(.*?)([^A-Za-z]*)$")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Also remove the silent fallback

I agree, and I found more. The comment on line 111 says that the pattern always matches, but it does not match a token with a line break. Lines 112-113 then return the plaintext word with no error, and # pragma: no cover hides this branch from coverage. CRLF has the same problem: line one\r\nline two becomes lubinube one\r\nline twubo, so the two words at each line break stay plaintext. On head, 'a' + '!' * n + 'b' takes about 0.9 s at n=10,000 and 14 s at n=40,000 on the event loop. Unspaced CJK text between ASCII letters is a realistic trigger: about 0.9 s at 10,000 characters and 3.7 s at 20,000. The preferred fix is to build the class on SearchReplaceConverter (see the comment on line 20), which removes this regex. If you keep the token parser, replace _TOKEN_PATTERN with a scan for the first and last ASCII letter, delete the guard and the pragma, and add tests for hello\nworld -> hubellubo\nwuborld and for a long non-letter run between two letters.

@richlundeen Richard Lundeen (richlundeen) left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Withdrawn: this earlier suggestion to drop the named converter in favor of configuring an existing one no longer applies. Named converters are welcome; see the current review.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation review at dd1a23d. The named UbbiDubbiConverter and its ALL-only ubbi_dubbi technique are welcome; these comments are about how the class is built (build it on SearchReplaceConverter, or one regex substitution on WordLevelConverter), the line-break and backtracking defect in the existing Copilot thread, the unused vowels option, and docs and test conventions.

Comment thread pyrit/converter/ubbi_dubbi_converter.py Outdated
_DEFAULT_VOWELS = "aeiou"


class UbbiDubbiConverter(WordLevelConverter):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Build the converter on SearchReplaceConverter

Keep the named class, but do not reimplement the substitution. The class adds a token regex, a vowel loop, and a casing layer (lines 13 and 80-129), but SearchReplaceConverter already expresses this as one regex substitution. The casing layer lowercases each mixed-case word, so the output does not decode to the input: iPhone becomes ubiphubonube and PyRIT becomes Pyrubit. In garak.encoding, 13 of the 17 default garak_web_html_js seeds do not decode back to the seed. For example, String.fromCharCode decodes to string.fromcharcode, and the base64 in the data: URL seed is lowercased, so it decodes to different bytes, not the script. The default DecodingScorer ignores case, so scores do not change, but the converter loses information that a letter-preserving cipher does not need to lose. Make the class a subclass of SearchReplaceConverter with a no-argument __init__ that calls super().__init__(pattern=r"([aeiou]+)", replace=r"ub\1", regex_flags=re.IGNORECASE). I tested this form: all 17 seeds round-trip, hello\nworld becomes hubellubo\nwuborld, and the identifier records the pattern, replacement, and flags. The output keeps the source case and inserts a lowercase ub (Apple becomes ubApplube). Then remove _TOKEN_PATTERN, _DEFAULT_VOWELS, _transform_core, the casing block, _build_identifier, and their tests, and use SelectiveTextConverter for word selection. If you must keep built-in word selection, keep WordLevelConverter and make convert_word_async one compiled-regex substitution that keeps the source letters, like ROT13Converter's one-line convert_word_async.

Comment thread pyrit/converter/ubbi_dubbi_converter.py Outdated
def __init__(
self,
*,
vowels: str = _DEFAULT_VOWELS,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Remove the vowels option

No caller sets vowels: the Encoding scenario and the doc example use the default, and only tests set it. The framework says not to add options for hypothetical needs, and a public parameter needs a deprecation to remove later. The option also does not do what the docstring says. The punctuation regex treats only ASCII letters as letters, so a non-ASCII vowel at a word edge never gets ub: UbbiDubbiConverter(vowels="aeiouéè") leaves été unchanged. Remove vowels, _DEFAULT_VOWELS, the empty-vowels ValueError, the _build_identifier override, the UbbiDubbiConverter.vowels identifier sample, and the four vowel tests. The SearchReplaceConverter change on line 20 also removes this option.

print("ROT13:", await ROT13Converter().convert_async(prompt=prompt)) # type: ignore
print("Base64:", await Base64Converter().convert_async(prompt=prompt)) # type: ignore
print("Base2048:", await Base2048Converter().convert_async(prompt=prompt)) # type: ignore
print("Ubbi Dubbi:", await UbbiDubbiConverter().convert_async(prompt=prompt)) # type: ignore

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Update the .ipynb pair of this notebook

This change adds the UbbiDubbiConverter import and print line to the .py file only. 1_text_to_text_converters.ipynb has no Ubbi Dubbi text, and the docs site shows the .ipynb. The framework says that each percent-format .py file stays in sync with its .ipynb pair, and docs.instructions.md says never to edit only one file of the pair. Add the same import and print line to the .ipynb code cell, as the BifidConverter PR (#4309) did.

LeetSpeak = ("leet_speak", {"default"})
AsciiSmuggler = ("ascii_smuggler", set[str]())
Bifid = ("bifid", set[str]())
UbbiDubbi = ("ubbi_dubbi", set[str]())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] List ubbi_dubbi in the garak scanner guide

This line adds the ubbi_dubbi technique, but doc/scanner/garak.py and garak.ipynb still say "Available techniques (18 encodings)" and end the list at Bifid. A reader of the scanner guide does not find ubbi_dubbi as a --techniques value. Add UbbiDubbi to the list in both files, and set the count to match the enum after you rebase. Main already has ColumnarTransposition (#4327), which is also missing from the list, so the count is 20 if no other new technique merges first. The BifidConverter PR (#4309) updated both files in the same way.

Comment thread pyrit/converter/__init__.py Outdated
from pyrit.converter.toxic_sentence_generator_converter import ToxicSentenceGeneratorConverter
from pyrit.converter.translation_converter import TranslationConverter
from pyrit.converter.transparency_attack_converter import TransparencyAttackConverter
from pyrit.converter.ubbi_dubbi_converter import UbbiDubbiConverter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] Add the converter to the modality table output

converters.instructions.md says that a new converter must update the modality table in doc/code/converters/0_converters.ipynb. The .py file builds the table from get_converter_modalities(), but the saved .ipynb output has no row for UbbiDubbiConverter, so the docs site does not show it. Add the UbbiDubbiConverter row to the saved table output in 0_converters.ipynb, as the FullwidthConverter PR (#4185) did. Running jupytext --execute on this notebook also works, but it needs OpenAI credentials because later cells call OpenAIChatTarget. BifidConverter (and ColumnarTranspositionConverter after you rebase) are missing too. Other open PRs in this series change the same output, so one update after the series merges is also acceptable if you say so in the PR.

from pyrit.converter import UbbiDubbiConverter


@pytest.mark.asyncio

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] Remove the asyncio marks and parametrize the tests

test.instructions.md says not to add @pytest.mark.asyncio, because asyncio_mode = "auto" is set for the project. This file adds the mark 18 times. Also, 14 tests have the same shape: make a converter, convert one input, and compare the output. The framework says to parametrize instead of copying tests. Remove the marks, put the input and output cases in one @pytest.mark.parametrize table, and keep the error and identifier tests separate.

@romanlutz

Copy link
Copy Markdown
Contributor Author

Reworked per your suggestion: dropped UbbiDubbiConverter; the ubbi_dubbi technique is now SearchReplaceConverter(pattern=r'([aeiouAEIOU]+)', replace=r'ub\1') (vowel-run prefix, source case preserved), keeping the named technique + a focused test. Thanks!

@romanlutz Roman Lutz (romanlutz) changed the title FEAT: Add UbbiDubbiConverter (ubbi_dubbi converter) FEAT: Add ubbi_dubbi Encoding technique via SearchReplaceConverter Nov 20, 2026
Per review (reuse and compose before adding): keep the ubbi_dubbi technique but implement it with SearchReplaceConverter instead of a dedicated UbbiDubbiConverter class.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up review at 3365164. The rework to one SearchReplaceConverter row in the Encoding scenario removes the causes of my earlier comments on the converter class, its docs, and its tests. One new P3 is on the new test. The earlier P3 about the garak scanner guide (r4638761720) still applies: doc/scanner/garak.py and garak.ipynb still list 18 encodings that end at Bifid.


async def test_ubbi_dubbi_technique_inserts_ub_before_vowels():
"""The ubbi_dubbi Encoding technique (a SearchReplaceConverter config) prefixes vowel runs with 'ub'."""
converter = SearchReplaceConverter(pattern=r"([aeiouAEIOU]+)", replace=r"ub\1")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] Test the registered ubbi_dubbi row

This test makes its own SearchReplaceConverter with a copy of the regex. Thus, it tests SearchReplaceConverter, not the new Encoding technique. If you delete the ubbi_dubbi row in encoding.py, or change its pattern so that it matches nothing, all tests still pass. I patched the row to pattern="unmatchable", and the 33 tests in the two changed files passed. Get the converter from Encoding instead, for example the ubbi_dubbi row from Encoding._converter_variants(), or the converter stack of the selected EncodingTechnique.UbbiDubbi attack as the base64 tests in test_encoding.py do. Then assert its output. Put the case in test_encoding.py with a small @pytest.mark.parametrize table, for example speak -> spubeak, queue -> qubueue, and HELLO -> HubELLubO, and delete this separate file.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed. No blocking concerns remain. This review records source coverage; merging is handled separately.

[P3] inline:4638761720: R5: doc/scanner/garak.py:139-141 and garak.ipynb:306-308 still say 'Available techniques (18 encodings) ... AsciiSmuggler, Bifid' and omit ubbi_dubbi, which this PR adds (ALL now has 20 techniques). The PR changes no docs. Supported, unfixed, P3 docs discoverability (operators can still select ubbi_dubbi by name); feedback/feedback.json R5. Thread is current and unresolved; follow-up review 5996808015 restates it.

[P3] inline:4645621954: C1: tests/unit/scenario/garak/test_ubbi_dubbi_technique.py imports only SearchReplaceConverter and copies the regex, so it cannot fail if the registered encoding.py:324 row is deleted or broken (reviewer mutation: 33 passed with the row patched to 'unmatchable'). The live row is currently correct (speak -> spubeak). Supported, unfixed, P3 test-quality improvement; feedback/feedback.json C1.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants