Repository navigation
FIX: keep the request when an in-flight target call is cancelled - #3796
Conversation
Cancelling an attack after it entered the objective target left nothing in memory: CancelledError is a BaseException, so it skipped both the normalizer's request persistence and the attack's ON_ERROR result. - PromptNormalizer persists the converted request (without a response) when a send is cancelled while the target may have it, then re-raises. A send_context that never reached the target persists nothing, matching the Exception path. - AttackStrategy persists an UNDETERMINED AttackResult carrying the new AttackCancellation marker when it is cancelled after it started sending to the objective target, then re-raises. - Persistence failures are logged and never replace the cancellation. - Scenario resume treats cancelled rows as pending, like ERROR rows, so a resumed scenario re-runs objectives that were cancelled mid-flight. Fixes microsoft#3733
…ncelled-target-call # Conflicts: # pyrit/scenario/core/scenario.py
…ncelled-target-call # Conflicts: # pyrit/scenario/core/scenario.py # tests/unit/prompt_normalizer/test_prompt_normalizer.py
microsoft#3643 records interrupted attempts as scenario conversation provenance and asserts that a cancel adds no AttackResult and leaves progress counts unchanged. The UNDETERMINED cancellation result conflicted with that, so remove AttackCancellation, the AttackStrategy CancelledError handler and the scenario-resume exclusion. What remains is the PromptNormalizer change: a send cancelled after reaching the target persists its request, with no response, before re-raising. The recorded conversation for an interrupted attempt now holds that request.
|
I narrowed this PR. The branch had been red since my last merge of #3643 (merged after I opened this) records interrupted attempts as scenario conversation provenance. Its tests in I dropped that half: One question for #3733: a cancelled |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Track target invocation for ordinary sends, drain cancellation-path writes, and cover supplied and reused send contexts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Kept the narrowed request-only behavior in e0e0026: cancellation still propagates and no synthetic AttackResult is added. Requests cancelled before target invocation no longer enter conversation history, and an in-flight request finishes saving even if cancellation repeats. Added regressions for supplied and reused history contexts; the affected cancellation, send, tracing and scenario-resume tests pass. Verified on published commit |
Richard Lundeen (richlundeen)
left a comment
There was a problem hiding this comment.
Automated review completed. No blocking concerns remain. This review records source coverage; merging is handled separately.
Description
Addresses the lost-request half of #3733. When a target call is cancelled after the target received the request, the request was lost from memory, because
PromptNormalizersaved it only on success or onException, andCancelledErroris aBaseException. Now the request is saved before the cancellation propagates, so the conversation shows what the target was sent.PromptNormalizer.send_prompt_asynchandlesasyncio.CancelledError:Exceptionbranch's rule: ifsend_contextsays the target was never invoked, nothing is saved.AttackResultis written for a cancelled attempt. This matches 6 - FEAT: Scan Results and Live Chats #3643, which records interrupted attempts as scenario conversation provenance, and the tests there that assert a cancel adds no attack result. Therecorded_conversationsentry for an interrupted attempt now points to a conversation that holds the in-flight request.PromptSendingAttackcancelled while the target is handling the request leaves 0 message pieces. After: it leaves[("user", "cancellation audit probe")], andCancelledErrorstill propagates.B prompt 5.This change was developed with AI assistance (Claude Code).