Skip to content

FIX: keep the request when an in-flight target call is cancelled - #3796

Merged
Richard Lundeen (richlundeen) merged 6 commits into
microsoft:mainfrom
adimalkar:fix/3733-persist-cancelled-target-call
Nov 11, 2026
Merged

Richard Lundeen (richlundeen) merged 6 commits into
microsoft:mainfrom
adimalkar:fix/3733-persist-cancelled-target-call

Conversation

@adimalkar

@adimalkar Aditya Malkar (adimalkar) commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Description

Addresses the lost-request half of #3733. When a target call is cancelled after the target received the request, the request was lost from memory, because PromptNormalizer saved it only on success or on Exception, and CancelledError is a BaseException. Now the request is saved before the cancellation propagates, so the conversation shows what the target was sent.

  • PromptNormalizer.send_prompt_async handles asyncio.CancelledError:
    • It saves the converted request, with no response since none came back, then re-raises.
    • It follows the Exception branch's rule: if send_context says the target was never invoked, nothing is saved.
    • If saving the request fails, the failure is logged and the cancellation still propagates.
  • No AttackResult is written for a cancelled attempt. This matches 6 - FEAT: Scan Results and Live Chats #3643, which records interrupted attempts as scenario conversation provenance, and the tests there that assert a cancel adds no attack result. The recorded_conversations entry for an interrupted attempt now points to a conversation that holds the in-flight request.
  • Before: a PromptSendingAttack cancelled while the target is handling the request leaves 0 message pieces. After: it leaves [("user", "cancellation audit probe")], and CancelledError still propagates.
  • Tests whose assertions changed because the in-flight request is now kept:
    • Crescendo cancellation: the conversation ends with the unanswered fifth request.
    • Crescendo scenario resume: the partial conversation ends with B prompt 5.
    • Copilot session cleanup: the request is saved there too, and the cleanup assertions are unchanged.

This change was developed with AI assistance (Claude Code).

Cancelling an attack after it entered the objective target left nothing in
memory: CancelledError is a BaseException, so it skipped both the
normalizer's request persistence and the attack's ON_ERROR result.

- PromptNormalizer persists the converted request (without a response)
  when a send is cancelled while the target may have it, then re-raises.
  A send_context that never reached the target persists nothing, matching
  the Exception path.
- AttackStrategy persists an UNDETERMINED AttackResult carrying the new
  AttackCancellation marker when it is cancelled after it started sending
  to the objective target, then re-raises.
- Persistence failures are logged and never replace the cancellation.
- Scenario resume treats cancelled rows as pending, like ERROR rows, so a
  resumed scenario re-runs objectives that were cancelled mid-flight.

Fixes microsoft#3733
Copilot AI balanced review requested due to automatic review settings November 1, 2026 18:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

…ncelled-target-call

# Conflicts:
#	pyrit/scenario/core/scenario.py
…ncelled-target-call

# Conflicts:
#	pyrit/scenario/core/scenario.py
#	tests/unit/prompt_normalizer/test_prompt_normalizer.py
microsoft#3643 records interrupted attempts as scenario conversation provenance and
asserts that a cancel adds no AttackResult and leaves progress counts
unchanged. The UNDETERMINED cancellation result conflicted with that, so
remove AttackCancellation, the AttackStrategy CancelledError handler and the
scenario-resume exclusion.

What remains is the PromptNormalizer change: a send cancelled after reaching
the target persists its request, with no response, before re-raising. The
recorded conversation for an interrupted attempt now holds that request.
@adimalkar Aditya Malkar (adimalkar) changed the title FIX: keep an audit record when an in-flight target call is cancelled FIX: keep the request when an in-flight target call is cancelled Nov 11, 2026
@adimalkar

Copy link
Copy Markdown
Contributor Author

I narrowed this PR. The branch had been red since my last merge of main, and that was my fault.

#3643 (merged after I opened this) records interrupted attempts as scenario conversation provenance. Its tests in tests/unit/backend/test_scenario_resume.py check that a cancel adds no AttackResult and leaves the progress counts unchanged. The UNDETERMINED cancellation result this PR added broke six of them. I hadn't run that directory after merging.

I dropped that half: AttackCancellation, the AttackStrategy handler and the scenario-resume exclusion are gone. The PR now only saves the in-flight request in PromptNormalizer when a send is cancelled after reaching the target. That sits alongside #3643: the recorded conversation for an interrupted attempt now contains the request the target actually received. I've updated the description to match.

One question for #3733: a cancelled attack.execute_async(...) run outside a scenario still leaves no attack-level record. Do you want something for that case, or is the saved request enough?

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Track target invocation for ordinary sends, drain cancellation-path writes, and cover supplied and reused send contexts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@richlundeen

Copy link
Copy Markdown
Contributor

conversation:6789394020

Kept the narrowed request-only behavior in e0e0026: cancellation still propagates and no synthetic AttackResult is added. Requests cancelled before target invocation no longer enter conversation history, and an in-flight request finishes saving even if cancellation repeats. Added regressions for supplied and reused history contexts; the affected cancellation, send, tracing and scenario-resume tests pass.

Verified on published commit e0e002692fff06364409aef13185c94bc645bbc5.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed. No blocking concerns remain. This review records source coverage; merging is handled separately.

@richlundeen
Richard Lundeen (richlundeen) added this pull request to the merge queue Nov 11, 2026
Merged via the queue into microsoft:main with commit 9bb084c Nov 11, 2026
62 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants