Repository navigation
FIX: Drain sibling scorers when a ChunkedRequestAttack scorer fails - #4178
Merged
Richard Lundeen (richlundeen) merged 1 commit intoNov 11, 2026
Conversation
ChunkedRequestAttack scored the combined chunks with a bare asyncio.gather, so when one scorer failed, its siblings kept running and could persist scores after the attack had already failed. Use gather_with_cleanup_async, as microsoft#2695 and microsoft#2856 did for the other scorer fan-outs, so unfinished scorers are cancelled and drained before the error propagates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are recorded for the change to cancel and drain sibling scorers before propagating a scoring error.
No open issues 🎉
What changed in this PR
This PR extends the shared scorer-cleanup behavior to ChunkedRequestAttack, ensuring unfinished sibling scorers are cancelled and drained before a scoring error propagates.
Changes:
- Replace plain
asyncio.gatherwithgather_with_cleanup_async. - Add a regression test verifying that an auxiliary scorer’s failure leaves the objective scorer cancelled and finished before the caller receives the error.
| File | Description |
|---|---|
tests/unit/executor/attack/multi_turn/test_chunked_request.py |
Add regression coverage for sibling scorer cancellation and completion. |
pyrit/executor/attack/multi_turn/chunked_request.py |
Use the shared cleanup helper for concurrent scoring. |
Review effort: Balanced
Justin Song (jsong468)
approved these changes
Nov 11, 2026
Richard Lundeen (richlundeen)
deleted the
richlundeen-chunked-request-scorer-cleanup
branch
November 11, 2026 04:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When one scorer fails during
ChunkedRequestAttackscoring, the other scorers should be cancelled and finish cleaning up before the error propagates. Today they keep running and can save scores after the attack has already failed.asyncio.gatherinChunkedRequestAttack._score_combined_value_asyncwithgather_with_cleanup_asyncfrompyrit/common/task_utils.py🤖 Generated with Claude Code