Repository navigation
feat(guardrails): report provider errors on guardrail validation results [AL-673] - #1956
Conversation
There was a problem hiding this comment.
🟢 Approval recommended
The additive metadata preserves existing failure handling, has focused regression coverage, and leaves no unresolved blocking findings.
No open issues 🎉
What changed in this PR
This PR adds error metadata to guardrail validation results so callers can distinguish service failures from actual violations while preserving existing fail-closed behavior.
Changes:
- Adds and exports
GuardrailValidationErrorTypewith an optionalerror_typeresult field. - Maps backend errors case-insensitively and logs warnings for errors and unknown results.
- Adds regression tests and updates package versions, dependency requirements, and lockfiles.
| File | Description |
|---|---|
packages/uipath/uv.lock |
Updates locked core and platform versions. |
packages/uipath-platform/uv.lock |
Updates locked core and platform versions. |
packages/uipath-platform/tests/services/test_guardrails_service.py |
Tests error mapping, warnings, and preserved failure behavior. |
packages/uipath-platform/src/uipath/platform/guardrails/_guardrails_service.py |
Classifies backend errors and attaches metadata while failing closed. |
packages/uipath-platform/pyproject.toml |
Bumps platform version and minimum core dependency. |
packages/uipath-core/uv.lock |
Updates the locked core version. |
packages/uipath-core/tests/guardrails/test_guardrails_models.py |
Tests enum values, defaults, aliases, and serialization. |
packages/uipath-core/src/uipath/core/guardrails/guardrails.py |
Defines the error enum and optional result field. |
packages/uipath-core/src/uipath/core/guardrails/__init__.py |
Exports the new error enum. |
packages/uipath-core/pyproject.toml |
Bumps core to version 0.5.34. |
Review effort: Balanced
…lts [AL-673] The guardrail validate API answers PROVIDER_ERROR, INTERNAL_ERROR or RATE_LIMIT_EXCEEDED when it could not evaluate the input, for example when an LLM-as-Judge model rejects the request. evaluate_guardrail turned these into VALIDATION_FAILED through its unknown-result fallback, so callers could not tell a real violation from a failed evaluation (SRE-687087: judge models that deprecate `temperature` answered 400 and 90 eval runs were blocked). GuardrailValidationResult gains an additive error_type (GuardrailValidationErrorType) that is set for those results. result stays validation_failed, so every existing VALIDATION_FAILED check keeps firing its action (fail closed); error_type only says why the validation did not complete. Error and unknown results are now logged as warnings. Bumps uipath-core to 0.5.34 and uipath-platform to 0.2.42 (requiring uipath-core>=0.5.34) and refreshes the uv.lock files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0086f38 to
bcc7c95
Compare
🚨 Heads up:
|
|
🚨 Heads up:
|



What changed?
The guardrail validate API answers
PROVIDER_ERROR,INTERNAL_ERRORorRATE_LIMIT_EXCEEDEDwhen it could not evaluate the input.evaluate_guardrailturned these intoVALIDATION_FAILEDthrough its unknown-result fallback, so callers could not tell a real violation from a validation that never ran.Prod impact (SRE-687087): on 2026-10-22, an LLM-as-Judge model rejected
temperaturewith a 400. 90 eval runs were reported as guardrail violations.Changes:
GuardrailValidationResult.error_type(aliaserrorType) uses a newGuardrailValidationErrorTypeenum:provider_error,internal_errororrate_limit_exceeded.Nonefor every other result.resultstaysvalidation_failed, so the guardrail still fails closed. No consumer changes are needed:== / != VALIDATION_FAILEDcheck, here and in uipath-langchain (middlewares, adapter, escalate action, guardrail nodes), still fires its action.GuardrailValidationResultTypewas rejected on purpose. Those same checks would have started letting runs through, and released uipath-langchain versions would pick up a newer platform.uipath.core.guardrailsonly. It is not added to theuipath.platform.guardrailsre-export block, because that block is marked for removal in 2.3.0.Versions:
uv.lockfiles are refreshed.Follow-ups:
Ticket: AL-673
How has this been tested?
error_typedefaults toNone.errorTypealias round-trips.validation_failed, the matchingerror_type, and the backenddetailskept asreason. This holds for lowercase results too.validation_failed,error_type=Noneand a warning.PASSEDandVALIDATION_FAILEDgiveerror_type=None.ruff check,ruff format --checkandmypyare clean on both packages.check_version_uniqueness.pyandcheck_dependency_version_bumps.pypass, anduv lock --lockedis current.🤖 Generated with Claude Code