Repository navigation
FEAT: Delete saved registry entries (part 20) - #4071
varunj-msft wants to merge 690 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Version-checked preset and suite deletes leave PermissionError unhandled, causing a 403 response that exposes server filesystem paths instead of a sanitized storage-unavailable 503.
2 open issues
What changed in this PR
Adds saved registry-entry deletion to CoPyRIT, building on persistence, ownership display, and restore recovery. The stacked changes also protect preset and suite deletes against stale versions.
Changes:
- Connects saved-entry management and restore recovery to backend routes and registry tables.
- Adds ownership, persistence eligibility, and protection against replacing saved entries.
- Updates version-checked deletion, isolated test storage, and user documentation.
| File | Description |
|---|---|
tests/unit/setup/test_targets_initializer.py |
Tests saved-target collision protection. |
tests/unit/setup/test_scorer_initializer.py |
Tests saved-scorer tagging protection. |
tests/unit/setup/test_converter_initializer.py |
Tests saved-converter replacement refusal. |
tests/unit/setup/test_configuration_loader.py |
Tests registry storage configuration. |
tests/unit/registry/test_scenario_suite_storage.py |
Tests stale suite-delete conflicts. |
tests/unit/registry/test_scenario_suite_registry.py |
Tests version-checked suite deletion. |
tests/unit/registry/test_scenario_preset_registry.py |
Tests version-checked preset deletion. |
tests/unit/registry/test_instance_registry.py |
Tests entry origins and replacement guards. |
tests/unit/registry/test_converter_registry.py |
Tests saved-name collision recording. |
tests/unit/registry/test_attack_technique_registry.py |
Tests saved-technique collision handling. |
tests/unit/backend/test_technique_edit_service.py |
Updates keyword-only service calls. |
tests/unit/backend/test_scenario_suite_service.py |
Tests delete-version forwarding. |
tests/unit/backend/test_scenario_suite_routes.py |
Tests delete conflicts and storage errors. |
tests/unit/backend/test_scenario_suite_execution_service.py |
Tests saved-entry launch conflict messages. |
tests/unit/backend/test_scenario_preset_service.py |
Tests stale preset-delete conflicts. |
tests/unit/backend/test_scenario_preset_routes.py |
Tests version checks and sanitized errors. |
tests/unit/backend/test_saved_entries.py |
Tests rejection of temporary dependencies. |
tests/unit/backend/test_mappers.py |
Tests origin and ownership mapping. |
tests/unit/backend/test_initializer_service.py |
Tests sanitized storage permission errors. |
tests/unit/backend/test_auth_middleware.py |
Tests current-user ID extraction. |
tests/unit/backend/test_analytics_benchmark.py |
Checks isolated recipe storage. |
tests/unit/backend/conftest.py |
Isolates saved entries in backend tests. |
tests/end_to_end/conftest.py |
Gives test backends fresh recipe storage. |
test_support/analytics_benchmark.py |
Configures benchmark recipe storage. |
pyrit/setup/initializers/tools.py |
Guards saved tool-entry collisions. |
pyrit/setup/initializers/targets.py |
Guards saved target-group collisions. |
pyrit/setup/initializers/scorers.py |
Prevents tagging saved substitutes. |
pyrit/setup/configuration_loader.py |
Adds registry recipe storage configuration. |
pyrit/score/true_false/local_refusal_classifier_scorer.py |
Declares persistence restriction. |
pyrit/score/true_false/audio_true_false_scorer.py |
Declares server-credential persistence restriction. |
pyrit/score/scorer.py |
Adds scorer persistence policy metadata. |
pyrit/score/float_scale/azure_content_filter_scorer.py |
Declares credential fallback behavior. |
pyrit/score/float_scale/audio_float_scale_scorer.py |
Declares audio scorer persistence restriction. |
pyrit/registry/scenario_suite_storage.py |
Supports conditional suite deletion. |
pyrit/registry/scenario_preset_storage.py |
Supports conditional preset deletion. |
pyrit/registry/components/scorer_registry.py |
Reserves the types instance name. |
pyrit/registry/components/scenario_suite_registry.py |
Forwards suite-delete versions. |
pyrit/registry/components/scenario_preset_registry.py |
Forwards preset-delete versions. |
pyrit/prompt_target/websocket_copilot_target.py |
Declares persistence restriction. |
pyrit/prompt_target/prompt_shield_target.py |
Declares credential and destination policies. |
pyrit/prompt_target/openai/openai_target.py |
Declares header, key, and endpoint policies. |
pyrit/prompt_target/litellm_chat_target.py |
Declares API-key fallback behavior. |
pyrit/prompt_target/hugging_face/hugging_face_chat_target.py |
Declares token and persistence restrictions. |
pyrit/prompt_target/http_target/http_target.py |
Disallows saving raw HTTP requests. |
pyrit/prompt_target/hack_a_prompt_target.py |
Declares session and cookie fallbacks. |
pyrit/prompt_target/github_copilot_target.py |
Declares token fallback behavior. |
pyrit/prompt_target/common/prompt_target.py |
Adds target persistence policy metadata. |
pyrit/prompt_target/azure_ml_chat_target.py |
Declares credential and endpoint policies. |
pyrit/prompt_target/azure_blob_storage_target.py |
Declares token and destination policies. |
pyrit/prompt_target/a2a_target.py |
Declares destination parameters. |
pyrit/models/catalog/target.py |
Adds target origin and ownership fields. |
pyrit/models/catalog/scorer.py |
Adds scorer origin and ownership fields. |
pyrit/models/catalog/__init__.py |
Exports recipe and restore models lazily. |
pyrit/converter/converter.py |
Adds credential fallback metadata. |
pyrit/converter/azure_speech_text_to_audio_converter.py |
Declares speech credential fallback. |
pyrit/converter/azure_speech_audio_to_text_converter.py |
Declares speech credential fallback. |
pyrit/backend/services/scenario_suite_service.py |
Passes conditional delete versions. |
pyrit/backend/services/scenario_suite_execution_service.py |
Exposes safe saved-entry conflict messages. |
pyrit/backend/services/scenario_run_service.py |
Reports initializer clashes with saved entries. |
pyrit/backend/services/scenario_preset_service.py |
Passes conditional delete versions. |
pyrit/backend/routes/techniques.py |
Adds ownership and persistence handling. |
pyrit/backend/routes/targets.py |
Adds ownership and persistence handling. |
pyrit/backend/routes/scorers.py |
Adds ownership and persistence handling. |
pyrit/backend/routes/scenarios.py |
Documents saved-entry launch conflicts. |
pyrit/backend/routes/scenario_suites.py |
Handles delete versions and storage errors. |
pyrit/backend/routes/scenario_presets.py |
Handles delete versions and storage errors. |
pyrit/backend/routes/restore_failures.py |
Exposes restore failures and retry. |
pyrit/backend/routes/initializers.py |
Sanitizes storage error reporting. |
pyrit/backend/routes/common.py |
Shares ownership and persistence-error helpers. |
pyrit/backend/models/techniques.py |
Adds persistence validation and ownership fields. |
pyrit/backend/models/targets.py |
Adds persistence validation and eligibility fields. |
pyrit/backend/models/scorers.py |
Adds persistence request validation. |
pyrit/backend/models/scenario_suites.py |
Documents delete-version use. |
pyrit/backend/models/scenario_presets.py |
Documents delete-version use. |
pyrit/backend/models/saved_entries.py |
Defines saved-file management responses. |
pyrit/backend/models/converters.py |
Adds persistence and ownership fields. |
pyrit/backend/middleware/runtime.py |
Admits recovery and saved-entry management routes. |
pyrit/backend/middleware/auth.py |
Exposes the current user's object ID. |
pyrit/backend/mappers/technique_mappers.py |
Maps technique origin and ownership. |
pyrit/backend/mappers/target_mappers.py |
Maps target origin and ownership. |
pyrit/backend/mappers/converter_mappers.py |
Maps converter origin and ownership. |
pyrit/backend/main.py |
Registers recovery and saved-entry routes. |
frontend/src/utils/registryOrigin.ts |
Shares origin labels and ownership captions. |
frontend/src/utils/registryName.ts |
Suggests valid, available registry names. |
frontend/src/utils/registryName.test.ts |
Tests registry-name validation and suggestions. |
frontend/src/test-utils/targetFixtures.ts |
Adds ownership and origin fixture fields. |
frontend/src/services/errors.ts |
Recognizes persistence refusal responses. |
frontend/src/services/errors.test.ts |
Tests persistence refusal detection. |
frontend/src/services/api.ts |
Adds management calls and delete versions. |
frontend/src/services/api.test.ts |
Tests versioned delete requests. |
frontend/src/hooks/useRuntime.styles.ts |
Styles restore-failure lists and delete controls. |
frontend/src/components/Techniques/TechniqueEditor.test.tsx |
Updates converter save-button expectations. |
frontend/src/components/ScenarioSuites/ScenarioSuiteLibrary.tsx |
Handles delete conflicts and restores focus. |
frontend/src/components/ScenarioSuites/ScenarioSuiteLibrary.test.tsx |
Tests conflict reload and focus behavior. |
frontend/src/components/ScenarioPresets/ScenarioPresetLibrary.tsx |
Handles delete conflicts and restores focus. |
frontend/src/components/ScenarioPresets/ScenarioPresetLibrary.test.tsx |
Tests conditional deletion and focus recovery. |
frontend/src/components/Registry/TechniqueRegistry.tsx |
Adds ownership filtering and saved deletion. |
frontend/src/components/Registry/ScorerRegistry.tsx |
Adds ownership filtering and saved deletion. |
frontend/src/components/Registry/ScorerRegistry.test.tsx |
Tests origins, ownership filters, and delete controls. |
frontend/src/components/Registry/ScenarioPresetRegistry.test.tsx |
Expects versioned preset deletion. |
frontend/src/components/Registry/SavedBy.tsx |
Renders shared ownership captions. |
frontend/src/components/Registry/SavedBy.styles.ts |
Styles ownership captions. |
frontend/src/components/Registry/OriginBadge.tsx |
Displays registry-entry origins. |
frontend/src/components/Registry/DeleteSavedEntryButton.styles.ts |
Styles delete-dialog content. |
frontend/src/components/Registry/ConverterRegistry.tsx |
Separates saved deletion from live removal. |
frontend/src/components/Converters/SaveConverterDialog.tsx |
Clarifies that converter copies are temporary. |
frontend/src/components/Converters/SaveConverterDialog.test.tsx |
Tests temporary-copy wording. |
frontend/src/components/Config/TargetTable.tsx |
Adds origins, ownership filtering, and deletion. |
frontend/src/components/Config/TargetTable.styles.ts |
Styles ownership and filter controls. |
frontend/src/components/Config/TargetConfig.tsx |
Refreshes targets after deletion. |
frontend/src/App.tsx |
Places the runtime banner inside the theme. |
frontend/playwright.config.ts |
Isolates seeded backend recipe storage. |
frontend/e2e/fixtures/scenario_dataset_backend.py |
Forwards delete versions through the storage guard. |
frontend/e2e/fixtures/manual_send_backend.py |
Uses temporary recipe storage. |
frontend/e2e/config.spec.ts |
Tests target saving and ownership filtering. |
frontend/e2e/_targets.ts |
Adds ownership and origin mock fields. |
doc/gui/0_gui.md |
Documents saving, ownership, deletion, and recovery. |
doc/getting_started/pyrit_conf.md |
Documents recipe storage and restore recovery. |
.pyrit_conf_example |
Adds registry recipe storage guidance. |
Review effort: Balanced
| status_code=status.HTTP_409_CONFLICT, | ||
| detail=f"Scenario preset '{name}' changed since it was read; re-read it and retry", | ||
| ) from None | ||
| except (AzureError, TimeoutError) as exc: |
| status_code=status.HTTP_409_CONFLICT, | ||
| detail=f"Scenario suite '{name}' changed since it was read; re-read it and retry", | ||
| ) from None | ||
| except (AzureError, TimeoutError) as exc: |
Its text can carry a SAS URL. The 503 handler no longer logs it as a debug traceback, and the scenario suite service no longer attaches it to its warning; both log the storage message, which has no credentials.
Its constructor imports torch and checks for CUDA, which can hold the loop for seconds the first time. The target create now does both in a worker thread first, so the constructor, its errors, and the model-loading task it starts stay on the loop and are quick.
|
Richard Lundeen (@richlundeen) one call I'd like you to confirm. With who saved an entry no longer used for access, deleting a saved entry here (and changing one in #4072) now needs an administrator, as for scenario presets and suites. In local mode that means |
…ich deleting a saved entry takes
… a saveability answer once its request goes away The private-target test now controls the debounce clock, so a slow run can't check the values chosen before Only me.
…s, and auth mode only
# Conflicts: # pyrit/backend/main.py # pyrit/backend/services/runtime_lifecycle.py
Signed-in users may create shared converters and scorers that connect with this server's credentials, so a saved one isn't refused for that. This takes out the refusals of the Azure Speech converters and the Content Safety, audio and local refusal scorers. A saved entry still can't hold a typed key.

Description
This is one of the changes for saving Registry entries across restarts. It builds on #3965 (the list of entries that couldn't be restored, merged) and the version-checked deletes from #3656, also on
main. Until the earlier PRs merge, the diff here also shows their changes.A saved target, converter, scorer or technique couldn't be deleted from the app. The only way was to delete its file under
registry_recipes_sourceand restart, even for an entry that failed to restore and kept the backend from being ready.GET /api/saved-entries/{kind}/{name}describes a saved file: its version, why it can't be used if it can't, and whether you may change it.DELETE /api/saved-entries/{kind}/{name}?expected_version=<version>deletes the file and removes the entry it restored.kindistargets,converters,scorersortechniques.PYRIT_ALLOW_UNAUTHENTICATED_ADMIN=true. Who saved an entry stays in its file as attribution only. This replaces the earlier owner-or-admin rule, following the review on FEAT: Show who saved each target and list only your own (part 10) #4028.DELETE /api/converters/{id}still refuses a saved converter (409), and its message points to the new route.What stays the same: saving and restoring work as before, except that a save waits for a delete in progress and checks again that what it names still comes back.
DELETE /api/converters/{id}still removes Temporary and startup converters from the live registry. Startup entries have no saved file, so they can't be deleted here (404).Screenshots: Delete on a saved target in the Targets table, and the dialog.


Not covered:
DELETE /api/converters/{id}.