Repository navigation
FIX: Refuse saved targets that would authenticate with a key (part 7) - #4025
varunj-msft merged 207 commits into
Conversation
…API input External construction now rejects a parameter whose type includes Path unless the calling service passes the name in allowed_path_params after making the value safe. The converter service allows its upload parameters once they are stored, and the target service allows the server-set upload folder and the GitHubCopilotTarget working directory. The scorer-only check is removed, and the "Valid parameters" list leaves out path parameters callers cannot set.
The target catalog now applies the path rule to required parameters too, so a target that needs a disallowed path is left out instead of listed without the field, matching the scorer catalog. allowed_path_params is typed frozenset[str] so a bare string cannot turn the allow-list into a substring match.
The target catalog still lists a target whose required path parameter is its server-set upload folder when the server has one, and leaves it out otherwise.
# Conflicts: # tests/unit/backend/test_converter_service.py
…benchmark's recipes folder
The runtime status keeps its short summary. The restore log now also lists each field's full location and message, escaped, without the stored values, and with URL credentials removed. Credentials are removed before a location is shortened, so the summary can't keep part of one either.
Its text can carry a SAS URL. The 503 handler no longer logs it as a debug traceback, and the scenario suite service no longer attaches it to its warning; both log the storage message, which has no credentials.
A suite launch runs its launch scripts during preflight and sends its scans without them, so the clash is raised there. Map it to 409 with its message, as a single launch does, and drop the per-entry message that a clash can no longer reach.
Richard Lundeen (richlundeen)
left a comment
There was a problem hiding this comment.
One nonblocking documentation comment from the source review.
Richard Lundeen (richlundeen)
left a comment
There was a problem hiding this comment.
Automated review completed. No blocking concerns remain. This review records source coverage; merging is handled separately.
[P3] inline:4558287987: This is the coordinator's publication of the already verified local DOC-1, not a new independent defect. publication-confirmed.json and snapshot-final.json agree on head d4337d8, doc/getting_started/pyrit_conf.md RIGHT line 205, root 4558287987, review 5884349355 and the exact verified P3 comment body. Reuse unchanged DOC-1 evidence: moving source authentication/default details before recipe-specific restrictions removes bounded setup discoverability friction. The source has not changed, so the supported issue remains open P3 and nonblocking. Canonical cause is the same local DOC-1; do not create another finding or repair obligation.
[P3] local:DOC-1: Independently supported and unfixed at doc/getting_started/pyrit_conf.md:197-205. Storage default and Blob authentication follow the newly expanded target recipe rule list at the end of changed line 205. An operator configuring the source must scan unrelated recipe-author requirements to find the shared setup details. Move those details beside the YAML example and give the requirements a short subheading, preserving accurate restrictions, shared-identity/admin warnings and recovery. This is bounded discoverability friction, not wrong behavior in a supported workflow, and remains nonblocking P3. Current RIGHT line 205 is present in the collected patch. See verification/feedback.json, DOC-1.
# Conflicts: # pyrit/prompt_target/common/prompt_target.py # pyrit/prompt_target/hugging_face/hugging_face_chat_target.py
Resolve conflicts with microsoft#4025 and other main changes by keeping both sides: - prompt targets keep caller_credential_parameter next to microsoft#4025's server_fallback_parameters, destination_parameters, server_header_variables and not_saveable_reason; - target creation uses main's TargetRegistry.create_instance_async, which now accepts the server-chosen server_params (caller credential provider); create_instance_from_external_input returns to main's signature; - OpenAIChatTarget keeps main's deepcopy of passthrough parameters and the token-provider header check; - backend models, README, framework.md and tests keep both sides' additions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Description
This is one of the changes for saving Registry entries across restarts. It builds on #3986, now merged.
Problem. Restore refuses a recipe that stores a credential, but a target can use one without storing it. An OpenAI target with an empty
api_keyreads the key from this server's environment, an emptyendpointtakes this server's endpoint setting (which can hold a user name and password), and a URL can carry a SAS token or a key in its query. Saved targets are shared by everyone using the app, so one restored like that would send every user's requests with this server's keys.Change. Restore now runs every saved entry through one shared check,
check_saved_entry_params. On top of the rules it already had (no stored credential, no class that needs the server's upload folder), it refuses a target recipe that:server_fallback_parameters, with the value that turns the fallback off (for example, OpenAI'sheadersset to"{}"). Microsoft Entra ID ("identity") and no key ("none") are fine. Entra ID uses this server's Azure identity the same way targets created through the API do today, and an OpenAI target only mints that token for a recognized Azure endpoint.destination_parameters. One that would take this server's setting must be a full http(s) or ws(s) address. None may hold a user name, password, SAS token, fragment,;,&or=in the path, or a query other thanapi-version, even after percent-decoding and Unicode normalization, and a path can't have a query at all.not_saveable_reason:WebSocketCopilotTarget,HTTPTargetandHuggingFaceChatTarget. (HTTPXAPITargetwas already refused, since it needs the server's upload folder.)Each refusal is reported as a restore failure whose message names the class or parameter, never a value.
What stays the same. Converters and scorers get the same checks as before, since they don't declare fallbacks or destinations yet. Targets created without saving, and targets the startup scripts register, aren't checked, so a saved round-robin can still use startup targets, which keep their own sign-in.
server_credential_parameter(the admin check from #3247) is unchanged.Not covered. Environment variables the OpenAI SDK reads by itself (
OPENAI_CUSTOM_HEADERS,OPENAI_ORG_ID,OPENAI_PROJECT_ID) still apply to every OpenAI target on this server, saved ones included.Tests and Documentation
tests/unit/registry/test_registry_recipe_rules.py: Entra ID and no-key targets pass; an OpenAI target without a key or identity, an empty key the class reads from this server, identity withoutheaders, headers that can carry a credential, an auth mode the class doesn't support, and a class that can't be saved are refused; plain addresses and A2A agent card paths pass; URLs and paths that can carry a credential (user info, SAS, fragment, path parameters, other query keys, encoded or look-alike forms, and values that never finish decoding) are refused; an empty destination that would take this server's setting is refused, and one that isn't a full address gets its own message; text parameters that aren't destinations are left alone. Each URL rule also has a case only it catches, such as;or&alone in a path,sig=orSIG=in the host, a loneapi-key,subscription-keyorsvquery, anapi-versionvalue that isn't version-shaped, and an agent endpoint with another scheme, no host, or a value that can't be parsed.tests/unit/registry/test_registry_recipe_restore.py: restore reports a target that would use this server's credentials, or sign in with this server's account, instead of building it. The test that fed a server endpoint holding a credential into a build error is gone, since such a target is now refused before it's built; the tests that build errors hide credentials stay.uv run --frozen --extra all python -m pytest -n 8 --dist=loadfile tests/unit/registry tests/unit/backend tests/unit/prompt_target tests/unit/models tests/unit/setup tests/unit/converter tests/unit/score: 19,447 passed, 9 skipped, 4 xfaileddoc/getting_started/pyrit_conf.md: what a saved target can hold and use, which addresses must be full, the OpenAI SDK settings that still apply, and which target types can't be saved yet.No JupyText changes.