Skip to content

FEAT: redacted argument matching, so a redacted value never decides a score - #4359

Open
WatchTree-19 wants to merge 3 commits into
microsoft:mainfrom
WatchTree-19:feat/redacted-argument-matching
Open

WatchTree-19 wants to merge 3 commits into
microsoft:mainfrom
WatchTree-19:feat/redacted-argument-matching

Conversation

@WatchTree-19

Copy link
Copy Markdown
Contributor

Description

This answers open question 2 in the scorer contract design: "How is a redacted argument matched?" It builds on argument matching (#4150).

Since #4150, trace evidence keeps no arguments at all, so an argument requirement scored from traces is always undetermined. Section 6.4 says redaction happens at write time and must be recorded, so that a matcher never mistakes redaction for absence. This PR adds the smallest piece that makes redaction real and answers the question for it.

Keeping arguments is opt-in.

OtelTraceSource(trace_client=client, kept_arguments=("to",))
  • The source reads each span's gen_ai.tool.call.arguments attribute (the GenAI semantic convention attribute for tool call arguments), keeps the named top-level arguments on ToolExecution.arguments, and drops every other argument, and the raw attribute, before the observation is stored.
  • ToolEventsObservationPayload.kept_arguments records the list, so a replay knows which arguments were redacted. The payload validator rejects an event that holds any argument outside it. arguments_retained stays False: raw arguments are still never stored.
  • With no kept arguments, the source identity, the payload and each event keep their stored form, and nothing about name-only scoring changes.

How a redacted argument is matched. A redacted argument is unknown, not absent. A requirement on it can neither confirm nor refute a call, so any verdict that would rest on it is undetermined, even when the trace evidence is complete. The rationale names the arguments the source did not keep. A source that keeps no arguments gives the rationale and warning it gives on main today. Concretely, with kept_arguments=("to",) and complete evidence:

Requirement on send_email Call arguments Verdict
to == a {"to": "a", ...} true
to == a {"to": "b", ...} false (rests on the kept to)
to == a no to false (a kept argument that is absent)
to == a, body == x {"to": "a", "body": "x"} undetermined (body was redacted)
to == a, body == x {"to": "b", ...} false (the kept to already refutes it)
body == x, plus lookup by name no lookup call false (rests on lookup being absent)

Arguments that are missing from the span, do not parse (including non-finite constants such as NaN), or are not a JSON object leave the call undetermined.

The design's "likely answer" is that a redacted field forces PARTIAL. This applies that per requirement rather than to the whole observation: the trace's coverage of tool names is still complete, so marking the observation PARTIAL would also turn name-only negatives undetermined on replay. Instead, a requirement on a redacted argument gets exactly the outcome partial evidence gives (no confident negative, and no positive it cannot show), while verdicts that do not rest on a redacted value keep their confidence. MessageToolCallScorer is unchanged: stored function_call pieces hold the full arguments, and it still never reports false.

Tests and Documentation

  • tests/unit/score/test_tool_argument_matching.py: a redacted argument can neither confirm nor refute a call, a kept argument the call lacks is absent rather than redacted, name-only requirements are unaffected, a negative that rests on redaction is undetermined.
  • tests/unit/score/test_otel_tool_call_scorer.py: the source keeps only the named arguments and stores no raw attribute, unreadable arguments are not retained and stay undetermined, the verdict table above through scoring and replay, redaction leaves name-only negatives confident, source identity records kept_arguments only when set, invalid kept_arguments are rejected.
  • tests/unit/models/test_tool_observation.py and tests/unit/models/test_trace.py: round trips with kept arguments, the stored form without them, rejection of arguments the payload does not declare kept, blank or repeated names, unstructured arguments.
  • doc/code/scoring/5_tool_call_scorer.py and .ipynb: how to keep arguments from traces and how a redacted argument is scored, kept in sync.

How it was tested

python -m pytest -q -n 4 tests/unit/score tests/unit/models tests/unit/common/test_lazy_package_imports.py
python -m ruff check <changed files>
python -m ruff format --check <changed files>
ty check <changed files>
  • tests/unit/score, tests/unit/models and the lazy import test: 10259 passed, 57 skipped.
  • ruff check and ruff format --check clean. ty reports no new diagnostics on the changed files compared with main.
  • The doc change is Markdown only: the .ipynb cells were regenerated from the .py cells and checked to match them, with outputs untouched.
  • Each guard was checked by breaking it and confirming tests fail, then restoring it:
Mutation Tests that fail
Remove the redaction check (a redacted argument reads as absent) 6
Let a redacted argument pass as matched 6
OTel scorer ignores payload.kept_arguments 3
Source keeps every argument instead of the named ones 7
Unparsable arguments become an empty object 2
Accept non-finite JSON constants 1
Non-object arguments become an empty object 4
Payload accepts arguments it does not declare kept 3
Payload accepts blank or repeated kept names 2
Source identity always records kept_arguments 1
Source constructor skips kept_arguments validation 4
Source does not record kept_arguments on the payload 10
Rationale no longer names redacted arguments 3
Payload always serializes kept_arguments 4
Event always serializes arguments 5

… score

OtelTraceSource gains kept_arguments, an opt-in list of top-level argument
names. It reads each span's gen_ai.tool.call.arguments attribute, keeps the
named arguments on ToolExecution.arguments, and drops the rest, and the raw
attribute, before the observation is stored. The payload records the list, and
its validator rejects an event that holds any other argument.

An argument the source did not keep was redacted at write time. A requirement
on it can neither confirm nor refute a call, so a verdict that would rest on it
is undetermined even when the trace evidence is complete, and the rationale
names the arguments that were not kept, or, when the source keeps none, says
that trace evidence keeps no arguments. A kept argument that the call lacks, or
whose value does not match, still refutes the call, and so does complete
evidence with no call of the tool. Arguments that are missing, do not parse, or
are not a JSON object leave the call undetermined.

With no kept arguments, the source identity, the payload and each event keep
their stored form.
Copilot AI balanced review requested due to automatic review settings November 17, 2026 16:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Non-finite numbers can bypass argument validation through numeric overflow or dictionary inputs, allowing confident verdicts when unreadable evidence should remain undetermined.

1 open issue
What changed in this PR

This PR extends tool argument matching from #4150 with opt-in argument retention for trace evidence. It treats redacted arguments as unknown while preserving confident verdicts supported by retained arguments or tool-name coverage.

Changes:

  • Adds argument retention settings and payload validation while preserving default serialization.
  • Updates trace scoring and rationales to account for redacted arguments.
  • Adds retention, matching, and replay tests, plus synchronized documentation.
File Description
tests/​unit/​score/​test_tool_argument_matching.py Tests redacted versus absent arguments.
tests/​unit/​score/​test_otel_tool_call_scorer.py Tests retention, scoring, replay, and source identity.
tests/​unit/​models/​test_trace.py Tests argument serialization and validation.
tests/​unit/​models/​test_tool_observation.py Tests retention declarations and payload compatibility.
pyrit/​score/​true_false/​otel_tool_call_scorer.py Uses retained arguments and explains redaction.
pyrit/​score/​tool_argument_matching.py Treats redacted arguments as unknown.
pyrit/​score/​observation/​otel_trace_source.py Adds opt-in argument parsing and retention.
pyrit/​models/​score/​trace.py Adds optional retained arguments to execution evidence.
pyrit/​models/​score/​observation.py Records and validates retained argument names.
doc/​code/​scoring/​5_tool_call_scorer.py Documents trace argument retention and scoring.
doc/​code/​scoring/​5_tool_call_scorer.ipynb Mirrors the documentation changes in notebook Markdown.

Review effort: Balanced

Comment on lines +56 to +58
if not isinstance(value, dict):
return None
return {name: value[name] for name in kept if name in value}
A literal too large to represent, such as 1e400, parses to infinity, and an
already-parsed arguments object skips parsing altogether, so both reached the
matcher. The source now rejects any NaN or infinite number, nested ones
included, on both paths, and the call stays undetermined.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants