Repository navigation
FIX: handle unmatched and nested document tags in PromptShieldTarget parser - #4469
Open
LX (RytterMohn) wants to merge 1 commit into
Open
LX (RytterMohn) wants to merge 1 commit into
LX (RytterMohn) wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are identified for the document-tag parsing changes.
No open issues 🎉
What changed in this PR
This PR hardens PromptShieldTarget document parsing, following up on #2838 to preserve prompt text when document tags are unmatched or nested.
Changes:
- Replaces split-based parsing with a linear scan that extracts matched document spans and preserves unmatched tags in
userPrompt. - Adds six regression cases covering stray closing tags, unclosed opening tags, literal opening tags inside documents, and empty documents.
| File | Description |
|---|---|
tests/unit/prompt_target/target/test_prompt_shield_target.py |
Adds parametrized regression tests for document-tag edge cases. |
pyrit/prompt_target/prompt_shield_target.py |
Updates parsing to preserve prompt text and extract matched document content. |
Review effort: Balanced
Author
|
@microsoft-github-policy-service agree |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix
PromptShieldTarget._input_parserso unmatched or nested<document>/</document>tags no longer misclassifyuserPromptcontent as documents or silently drop trailing prompt text (following up on #2838).split("<document>")/split("</document>")loop inpyrit/prompt_target/prompt_shield_target.pywith a linearstr.findscan that extracts only matched<document>...</document>spans.</document>tags, and unclosed<document>tags verbatim inuserPrompt, while keeping literal<document>occurrences inside a closed document indocuments.tests/unit/prompt_target/target/test_prompt_shield_target.pycovering stray closing tags, extra closing tags after documents, unclosed opening tags, literal<document>inside documents, and empty documents.Before / after examples:
hello </document> worlduserPrompt=" world",documents=["hello "]userPrompt="hello </document> world",documents=[]<document>doc1</document> mid </document> tailuserPrompt=" mid ",documents=["doc1"](taildropped)userPrompt=" mid </document> tail",documents=["doc1"]Explain the <document> taguserPrompt="Explain the tag",documents=[]userPrompt="Explain the <document> tag",documents=[]Summarize: <document>Here is a <document> tag</document> pleaseuserPrompt="Summarize: Here is a please",documents=[" tag"]userPrompt="Summarize: please",documents=["Here is a <document> tag"]