Repository navigation
FIX: Count jailbroken responses once per response in FuzzerGenerator (#4230) - #4317
Conversation
…icrosoft#4230) Ensure FuzzerGenerator._process_scoring_results attributes jailbreak verdicts per target response rather than per passing score category, preventing duplicate conversation IDs and premature goal completion when multi-score evaluators are used. Update MCTS reward calculation to divide jailbreaks by the count of target responses. Signed-off-by: hrushikeshramilla <ramillahrushikesh@gmail.com>
There was a problem hiding this comment.
🟡 Changes recommended
A nullable conversation ID causes a type error in the required ty check, and the execution test lacks assertions to catch an incorrect MCTS reward calculation.
2 open issues
What changed in this PR
Fixes #4230 by making FuzzerGenerator count successful responses rather than passing score categories, preventing premature goal completion with multi-category scorers.
Changes:
- Deduplicates jailbreak counts and conversation tracking by response piece.
- Calculates MCTS rewards using the number of responses rather than scores.
- Adds unit regressions for multi-category scoring, missing conversation IDs, and goal-based termination.
| File | Description |
|---|---|
tests/unit/executor/promptgen/fuzzer/test_fuzzer.py |
Adds response-level counting and execution regression tests. |
pyrit/executor/promptgen/fuzzer/fuzzer.py |
Deduplicates successful responses and corrects the reward denominator. |
Review effort: Balanced
| for piece in response_pieces: | ||
| if str(piece.id) in jailbroken_piece_ids: | ||
| jailbreak_count += 1 | ||
| context.jailbreak_conversation_ids.append(piece.conversation_id) |
There was a problem hiding this comment.
Narrowed piece.conversation_id explicitly with a None check before appending to context.jailbreak_conversation_ids, ensuring the attribute type is properly narrowed for type checkers.
| with ( | ||
| patch.object(target, "_send_prompt_to_target_async", new_callable=AsyncMock, side_effect=respond_async), | ||
| patch.object( | ||
| converter, | ||
| "convert_async", | ||
| new_callable=AsyncMock, | ||
| return_value=ConverterResult(output_text="{{ prompt }}", output_type="text"), | ||
| ), | ||
| ): | ||
| result = await generator.execute_async( | ||
| prompts=["objective"], | ||
| prompt_templates=["{{ prompt }}"], | ||
| max_query_limit=10, | ||
| ) |
There was a problem hiding this comment.
Added verification wrapping _mcts_explorer.update_rewards to assert that each successful query's MCTS reward is recorded at the true response-level success rate (reward == 1.0) across all queries.
…uzzer execution test Signed-off-by: hrushikeshramilla <ramillahrushikesh@gmail.com>
Varun Poojari (Varunpoojari)
left a comment
There was a problem hiding this comment.
The fix for multi-category scorers inflating jailbreak_count makes sense, and the end-to-end test asserting reward == 1.0 is a good regression guard. One question about units:
- Per piece vs. per response.
_process_scoring_resultsnow counts per entry inresponse_pieces, while_update_mcts_rewardsis givennum_queries=len(responses). Ifresponse_piecesis flattened fromresponses, a single message with two jailbroken pieces would be counted twice, andreward = jailbreak_count / num_queriescould exceed 1, which the comment says it must not. The title says "once per response". Should the counting group by message (or conversation) instead? I haven't seen howresponse_piecesis built, so I may be missing a guarantee of one piece per response. - Tests and docs. Please add a multi-piece message case to the tests. The docstring should also state that any passing score on a response counts it as a jailbreak, since that is a semantic choice for multi-category scorers.
…multi-piece test Signed-off-by: hrushikeshramilla <ramillahrushikesh@gmail.com>
|
Thank you Varun Poojari (@Varunpoojari) for the thorough review and great catch on the per-piece vs. per-response distinction! Updated in commit
|
Varun Poojari (Varunpoojari)
left a comment
There was a problem hiding this comment.
Thanks for the quick update. I read the diff at d474d92 and both of my earlier points look addressed:
- Per-response counting. Grouping by
response.message_piecesand checkingany(str(piece.id) in jailbroken_piece_ids ...)means a multi-piece message adds at most 1 tojailbreak_count, sojailbreak_count / len(responses)stays within [0, 1]. The newtest_process_scoring_results_multi_piece_response_counts_oncecovers exactly this case. - Docstring. The "any passing score attributes a jailbreak to the response" semantics are now stated in
_process_scoring_results.
One small optional point, from the diff only, so I may be missing a guarantee elsewhere: the conversation_id is None check inside the first loop now only validates the scored piece's id, while the id that is recorded comes from response.message_pieces[0] in the second loop. If pieces in one message always share a conversation_id this is moot. Otherwise it may be simpler to do the None check only where the id is recorded, or to record the id of the jailbroken piece. Also, a score whose message_piece_id matches no response piece would raise "Response piece has no conversation_id", which could be misleading for that case, though that behaviour predates this PR.
I haven't run the tests locally.
…idate where recorded Signed-off-by: hrushikeshramilla <ramillahrushikesh@gmail.com>
|
Thank you Varun Poojari (@Varunpoojari)! Adopted your suggestion in commit d6a08a9:
All 53 unit tests in est_fuzzer.py and formatting/lint checks pass cleanly. |


Fixes #4230.
Description
When multi-score evaluators (such as
AzureContentFilterScorer, which returns scores across multiple harm categories per response) were used withFuzzerGenerator,_process_scoring_resultsincrementedjailbreak_countand appended the response'sconversation_idfor every passing score category. Consequently:target_jailbreak_goal_countdistinct jailbroken responses were obtained.FuzzerResult.jailbreak_conversation_ids._step_async,_update_mcts_rewardswas passednum_queries=len(scores)rather than the number of responses, measuring the fraction of score categories passed rather than the jailbreak success rate across the question batch.This PR fixes these issues:
message_piece_id), attributing at most one jailbreak verdict per target response.conversation_idexactly once incontext.jailbreak_conversation_ids._step_asyncto use the number of response queries (len(responses)), ensuring the reward represents the proportion of target responses that achieved a jailbreak (bounded in[0, 1]).