Skip to content

FIX: Drain sibling scorers when a ChunkedRequestAttack scorer fails - #4178

Merged
Richard Lundeen (richlundeen) merged 1 commit into
microsoft:mainfrom
richlundeen:richlundeen-chunked-request-scorer-cleanup
Nov 11, 2026
Merged

Richard Lundeen (richlundeen) merged 1 commit into
microsoft:mainfrom
richlundeen:richlundeen-chunked-request-scorer-cleanup

Conversation

@richlundeen

Copy link
Copy Markdown
Contributor

Description

When one scorer fails during ChunkedRequestAttack scoring, the other scorers should be cancelled and finish cleaning up before the error propagates. Today they keep running and can save scores after the attack has already failed.

🤖 Generated with Claude Code

ChunkedRequestAttack scored the combined chunks with a bare asyncio.gather,
so when one scorer failed, its siblings kept running and could persist
scores after the attack had already failed. Use gather_with_cleanup_async,
as microsoft#2695 and microsoft#2856 did for the other scorer fan-outs, so unfinished scorers
are cancelled and drained before the error propagates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 11, 2026 00:26
@richlundeen
Richard Lundeen (richlundeen) marked this pull request as ready for review November 11, 2026 00:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the change to cancel and drain sibling scorers before propagating a scoring error.

No open issues 🎉

What changed in this PR

This PR extends the shared scorer-cleanup behavior to ChunkedRequestAttack, ensuring unfinished sibling scorers are cancelled and drained before a scoring error propagates.

Changes:

  • Replace plain asyncio.gather with gather_with_cleanup_async.
  • Add a regression test verifying that an auxiliary scorer’s failure leaves the objective scorer cancelled and finished before the caller receives the error.
File Description
tests/​unit/​executor/​attack/​multi_turn/​test_chunked_request.py Add regression coverage for sibling scorer cancellation and completion.
pyrit/​executor/​attack/​multi_turn/​chunked_request.py Use the shared cleanup helper for concurrent scoring.

Review effort: Balanced

@richlundeen
Richard Lundeen (richlundeen) added this pull request to the merge queue Nov 11, 2026
Merged via the queue into microsoft:main with commit f36bda4 Nov 11, 2026
61 checks passed
@richlundeen
Richard Lundeen (richlundeen) deleted the richlundeen-chunked-request-scorer-cleanup branch November 11, 2026 04:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants