Repository navigation
FEAT: redacted argument matching, so a redacted value never decides a score - #4359
Open
WatchTree-19 wants to merge 3 commits into
Open
WatchTree-19 wants to merge 3 commits into
WatchTree-19 wants to merge 3 commits into
Conversation
… score OtelTraceSource gains kept_arguments, an opt-in list of top-level argument names. It reads each span's gen_ai.tool.call.arguments attribute, keeps the named arguments on ToolExecution.arguments, and drops the rest, and the raw attribute, before the observation is stored. The payload records the list, and its validator rejects an event that holds any other argument. An argument the source did not keep was redacted at write time. A requirement on it can neither confirm nor refute a call, so a verdict that would rest on it is undetermined even when the trace evidence is complete, and the rationale names the arguments that were not kept, or, when the source keeps none, says that trace evidence keeps no arguments. A kept argument that the call lacks, or whose value does not match, still refutes the call, and so does complete evidence with no call of the tool. Arguments that are missing, do not parse, or are not a JSON object leave the call undetermined. With no kept arguments, the source identity, the payload and each event keep their stored form.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Non-finite numbers can bypass argument validation through numeric overflow or dictionary inputs, allowing confident verdicts when unreadable evidence should remain undetermined.
1 open issue
What changed in this PR
This PR extends tool argument matching from #4150 with opt-in argument retention for trace evidence. It treats redacted arguments as unknown while preserving confident verdicts supported by retained arguments or tool-name coverage.
Changes:
- Adds argument retention settings and payload validation while preserving default serialization.
- Updates trace scoring and rationales to account for redacted arguments.
- Adds retention, matching, and replay tests, plus synchronized documentation.
| File | Description |
|---|---|
tests/unit/score/test_tool_argument_matching.py |
Tests redacted versus absent arguments. |
tests/unit/score/test_otel_tool_call_scorer.py |
Tests retention, scoring, replay, and source identity. |
tests/unit/models/test_trace.py |
Tests argument serialization and validation. |
tests/unit/models/test_tool_observation.py |
Tests retention declarations and payload compatibility. |
pyrit/score/true_false/otel_tool_call_scorer.py |
Uses retained arguments and explains redaction. |
pyrit/score/tool_argument_matching.py |
Treats redacted arguments as unknown. |
pyrit/score/observation/otel_trace_source.py |
Adds opt-in argument parsing and retention. |
pyrit/models/score/trace.py |
Adds optional retained arguments to execution evidence. |
pyrit/models/score/observation.py |
Records and validates retained argument names. |
doc/code/scoring/5_tool_call_scorer.py |
Documents trace argument retention and scoring. |
doc/code/scoring/5_tool_call_scorer.ipynb |
Mirrors the documentation changes in notebook Markdown. |
Review effort: Balanced
Comment on lines
+56
to
+58
| if not isinstance(value, dict): | ||
| return None | ||
| return {name: value[name] for name in kept if name in value} |
A literal too large to represent, such as 1e400, parses to infinity, and an already-parsed arguments object skips parsing altogether, so both reached the matcher. The source now rejects any NaN or infinite number, nested ones included, on both paths, and the call stays undetermined.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Description
This answers open question 2 in the scorer contract design: "How is a redacted argument matched?" It builds on argument matching (#4150).
Since #4150, trace evidence keeps no arguments at all, so an argument requirement scored from traces is always undetermined. Section 6.4 says redaction happens at write time and must be recorded, so that a matcher never mistakes redaction for absence. This PR adds the smallest piece that makes redaction real and answers the question for it.
Keeping arguments is opt-in.
gen_ai.tool.call.argumentsattribute (the GenAI semantic convention attribute for tool call arguments), keeps the named top-level arguments onToolExecution.arguments, and drops every other argument, and the raw attribute, before the observation is stored.ToolEventsObservationPayload.kept_argumentsrecords the list, so a replay knows which arguments were redacted. The payload validator rejects an event that holds any argument outside it.arguments_retainedstaysFalse: raw arguments are still never stored.How a redacted argument is matched. A redacted argument is unknown, not absent. A requirement on it can neither confirm nor refute a call, so any verdict that would rest on it is undetermined, even when the trace evidence is complete. The rationale names the arguments the source did not keep. A source that keeps no arguments gives the rationale and warning it gives on main today. Concretely, with
kept_arguments=("to",)and complete evidence:send_emailto == a{"to": "a", ...}to == a{"to": "b", ...}to)to == atoto == a,body == x{"to": "a", "body": "x"}bodywas redacted)to == a,body == x{"to": "b", ...}toalready refutes it)body == x, pluslookupby namelookupcalllookupbeing absent)Arguments that are missing from the span, do not parse (including non-finite constants such as
NaN), or are not a JSON object leave the call undetermined.The design's "likely answer" is that a redacted field forces
PARTIAL. This applies that per requirement rather than to the whole observation: the trace's coverage of tool names is still complete, so marking the observationPARTIALwould also turn name-only negatives undetermined on replay. Instead, a requirement on a redacted argument gets exactly the outcome partial evidence gives (no confident negative, and no positive it cannot show), while verdicts that do not rest on a redacted value keep their confidence.MessageToolCallScoreris unchanged: storedfunction_callpieces hold the full arguments, and it still never reports false.Tests and Documentation
tests/unit/score/test_tool_argument_matching.py: a redacted argument can neither confirm nor refute a call, a kept argument the call lacks is absent rather than redacted, name-only requirements are unaffected, a negative that rests on redaction is undetermined.tests/unit/score/test_otel_tool_call_scorer.py: the source keeps only the named arguments and stores no raw attribute, unreadable arguments are not retained and stay undetermined, the verdict table above through scoring and replay, redaction leaves name-only negatives confident, source identity recordskept_argumentsonly when set, invalidkept_argumentsare rejected.tests/unit/models/test_tool_observation.pyandtests/unit/models/test_trace.py: round trips with kept arguments, the stored form without them, rejection of arguments the payload does not declare kept, blank or repeated names, unstructured arguments.doc/code/scoring/5_tool_call_scorer.pyand.ipynb: how to keep arguments from traces and how a redacted argument is scored, kept in sync.How it was tested
tests/unit/score,tests/unit/modelsand the lazy import test: 10259 passed, 57 skipped.ruff checkandruff format --checkclean.tyreports no new diagnostics on the changed files compared with main..ipynbcells were regenerated from the.pycells and checked to match them, with outputs untouched.payload.kept_argumentskept_argumentskept_argumentsvalidationkept_argumentson the payloadkept_argumentsarguments