Skip to content

FIX: handle unmatched and nested document tags in PromptShieldTarget parser - #4469

Open
LX (RytterMohn) wants to merge 1 commit into
microsoft:mainfrom
RytterMohn:fix/prompt-shield-unmatched-document-tags
Open

LX (RytterMohn) wants to merge 1 commit into
microsoft:mainfrom
RytterMohn:fix/prompt-shield-unmatched-document-tags

Conversation

@RytterMohn

Copy link
Copy Markdown

Description

Fix PromptShieldTarget._input_parser so unmatched or nested <document> / </document> tags no longer misclassify userPrompt content as documents or silently drop trailing prompt text (following up on #2838).

  • Replaced the nested split("<document>") / split("</document>") loop in pyrit/prompt_target/prompt_shield_target.py with a linear str.find scan that extracts only matched <document>...</document> spans.
  • Preserved unmatched closing tags, trailing text after extra </document> tags, and unclosed <document> tags verbatim in userPrompt, while keeping literal <document> occurrences inside a closed document in documents.
  • Added parametrized regression tests in tests/unit/prompt_target/target/test_prompt_shield_target.py covering stray closing tags, extra closing tags after documents, unclosed opening tags, literal <document> inside documents, and empty documents.

Before / after examples:

Input Before After
hello </document> world userPrompt=" world", documents=["hello "] userPrompt="hello </document> world", documents=[]
<document>doc1</document> mid </document> tail userPrompt=" mid ", documents=["doc1"] ( tail dropped) userPrompt=" mid </document> tail", documents=["doc1"]
Explain the <document> tag userPrompt="Explain the tag", documents=[] userPrompt="Explain the <document> tag", documents=[]
Summarize: <document>Here is a <document> tag</document> please userPrompt="Summarize: Here is a please", documents=[" tag"] userPrompt="Summarize: please", documents=["Here is a <document> tag"]

Copilot AI balanced review requested due to automatic review settings November 20, 2026 09:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are identified for the document-tag parsing changes.

No open issues 🎉

What changed in this PR

This PR hardens PromptShieldTarget document parsing, following up on #2838 to preserve prompt text when document tags are unmatched or nested.

Changes:

  • Replaces split-based parsing with a linear scan that extracts matched document spans and preserves unmatched tags in userPrompt.
  • Adds six regression cases covering stray closing tags, unclosed opening tags, literal opening tags inside documents, and empty documents.
File Description
tests/​unit/​prompt_target/​target/​test_prompt_shield_target.py Adds parametrized regression tests for document-tag edge cases.
pyrit/​prompt_target/​prompt_shield_target.py Updates parsing to preserve prompt text and extract matched document content.

Review effort: Balanced

@RytterMohn

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants