Repository navigation
FEAT: Restore saved registry entries at startup and stay not ready if any fail (part 4) - #3909
Merged
varunj-msft merged 136 commits intoNov 12, 2026
Conversation
…API input External construction now rejects a parameter whose type includes Path unless the calling service passes the name in allowed_path_params after making the value safe. The converter service allows its upload parameters once they are stored, and the target service allows the server-set upload folder and the GitHubCopilotTarget working directory. The scorer-only check is removed, and the "Valid parameters" list leaves out path parameters callers cannot set.
The target catalog now applies the path rule to required parameters too, so a target that needs a disallowed path is left out instead of listed without the field, matching the scorer catalog. allowed_path_params is typed frozenset[str] so a bare string cannot turn the allow-list into a substring match.
The target catalog still lists a target whose required path parameter is its server-set upload folder when the server has one, and leaves it out otherwise.
# Conflicts: # tests/unit/backend/test_converter_service.py
…benchmark's recipes folder
Contributor
Author
|
Thanks Rich, no worries! It's back to not ready, with your follow-up included:
Since one bad file now keeps the app down, saving also refuses a couple more things restore would (#4026, #4066), and editing a broken entry just saves its file (#4072). On your #3908 note about converters: #4051 already refuses saving a converter that sets any file parameter, so saving and restore can't disagree there. |
# Conflicts: # pyrit/backend/middleware/error_handlers.py # pyrit/backend/routes/scenario_presets.py # pyrit/backend/routes/scenario_suites.py # pyrit/registry/file_document_storage.py
…y the other storage tests do
Richard Lundeen (richlundeen)
approved these changes
Nov 11, 2026
Richard Lundeen (richlundeen)
left a comment
Contributor
There was a problem hiding this comment.
Approving. Please address the P3 on schema-validation diagnostics before merging; that change doesn't need another review.
The runtime status keeps its short summary. The restore log now also lists each field's full location and message, escaped, without the stored values, and with URL credentials removed. Credentials are removed before a location is shortened, so the summary can't keep part of one either.
Its text can carry a SAS URL. The 503 handler no longer logs it as a debug traceback, and the scenario suite service no longer attaches it to its warning; both log the storage message, which has no credentials.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This is one of the changes for saving Registry entries across restarts. It builds on #3908. Until the earlier PRs merge, the diff here also shows their changes.
Problem. Saved entries need to come back at startup and after a live apply. A broken one must stop runs instead of vanishing; otherwise a scan that expects a saved converter or
adversarial_chatwould run with something else, or fail later.Change.
registry_recipes_source: a local folder or Azure Blob location, default~/.pyrit/registry_recipes. It's validated likescenario_presets_sourceand documented inpyrit_conf.mdand.pyrit_conf_example.RuntimeLifecyclerestores saved entries at startup, after the initializers run, and on apply, after the registries are reset. The files are read in a worker thread, and the entries are built on the runtime loop, because a constructor such asHuggingFaceChatTarget's starts a task on it.pyrit/backend/services/saved_entry_restore.pycalls the target, converter, scorer and technique services), so a saved entry passes the same checks as a new one. Restore runs as the server and can't tell who wrote a file, so targets are built as an administrator would build them, with this server's upload folder.failedwith outcomerestore-failedand doesn't become ready. The readiness gate already answers 503 on every route except configuration, initializers, auth, health and runtime status, and/api/healthshowsready: false, so only the configuration pages work until it's fixed. A startup script that fails already works this way.failed. It resets the registries and restores again, so an administrator can fix or delete the listed files and apply instead of restarting. The failure message says to apply or restart when live apply is on, and to restart when it's off; applying with live apply off says to restart.GET /api/config/runtimelistsrestore_failuresandrestore_ignored.What stays the same. With no saved entries, startup and apply behave as before. A file whose name isn't a valid entry name is listed as ignored and doesn't block readiness. Apply still refuses
restart-required.Next. #3965 (part 5) lists the entries that couldn't be restored on the Configuration page, for administrators, and shows everyone else one line about it. #4071 (part 20) adds Delete to that list.
Tests and Documentation
tests/unit/backend/test_saved_entry_restore.py(new): each kind restores through its create; a target its create refuses fails only that entry; a target that uploads files gets this server's upload folder, and fails where the server sets none; a technique its create request refuses is reported as invalid.tests/unit/backend/test_runtime_lifecycle.py: startup restores an entry that uses a target the startup scripts register; the files are read off the runtime loop and the entries are built on it; an apply after a startup restore brings back the same converter and scorer, with the real registry reset; an entry that can't be restored keeps the app not ready, with 503 outside configuration andready: falsein/api/health; apply fromfailedrestores again once the file is fixed, and with live apply off it says to restart; each failure is logged once without URL credentials; an unreadable folder gets its own message; ignored files don't count as failures; fixing a file and restarting restores it; cancellation still propagates.tests/unit/registry/test_registry_recipe_restore.py: a file that fails validation logs every problem in full in its one error record, including many unknown fields and long keys that share a prefix, escaped, without the stored values or URL credentials, and the short summary has no URL credentials either.frontend/src/components/Configuration/Reinitialize.test.tsx: Reinitialize stays available while a restore failed.tests/unit/setup/test_configuration_loader.pycovers the new setting.pytest tests/unit/backend tests/unit/registry tests/unit/setup tests/unit/common -n 4: 6,383 passed, 9 skipped, 4 xfailed.jest src/hooks src/App.test.tsx src/components/Layout src/components/Registry src/components/Configuration: 820 passed.tsc --noEmitpassed. After the validation-log change:pytest tests/unit/registry tests/unit/backend -n 8: 5,326 passed, 9 skipped, 4 xfailed.doc/getting_started/pyrit_conf.mdand.pyrit_conf_exampledescriberegistry_recipes_source, who should be able to write to it, and that the backend doesn't become ready until every saved entry restores: fix or delete the listed files, then apply or restart. JupyText: not run (no notebooks changed).