Skip to content

fix(network): align REST and WebSocket endpoint handling - #5336

Merged
johntmyers merged 1 commit into
mainfrom
fix/rest-protocol-consistency/johntmyers
Nov 6, 2026
Merged

johntmyers merged 1 commit into
mainfrom
fix/rest-protocol-consistency/johntmyers

Conversation

@johntmyers

Copy link
Copy Markdown
Contributor

Summary

Keep REST endpoints on HTTP request handling and require explicit WebSocket endpoints for WebSocket upgrades. Apply consistent validation for credential-rewrite settings across policies, provider profiles, and runtime configuration.

Changes

  • Use the same REST upgrade response across relay and forwarding paths.
  • Restrict WebSocket credential rewriting to WebSocket endpoints, including consistent validation of mixed-case protocol names.
  • Update policy documentation, migration guidance, and regression coverage.

Release Notes

REST endpoints return 403 for HTTP upgrade requests in both enforce and audit modes. Configure WebSocket traffic with protocol: websocket and explicit handshake and message rules. websocket_credential_rewrite: true requires a WebSocket endpoint.

Testing

  • 1,999 library tests passed; two tests ignored.
  • Scoped Clippy, Rust formatting, and documentation checks passed.
  • Repository pre-commit checks passed.
  • Four Docker E2E tests passed: REST endpoint handling and WebSocket conformance.

Checklist

  • Follows Conventional Commits.
  • Commit is signed off (DCO).
  • Relevant documentation and policy-generation guidance updated.

Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Nov 6, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

@johntmyers
johntmyers marked this pull request as ready for review November 6, 2026 15:37
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. policy — No explicit risk acceptance found

REST endpoints previously relayed allowed WebSocket upgrades and supported websocket_credential_rewrite: true; the candidate refuses upgrades and rejects that configuration.

  • Impact: Existing WebSocket clients receive HTTP 403 even in audit mode, while affected sandbox policies, provider profiles, and stored policy revisions fail validation.
  • Action: Preserve REST upgrade and credential-rewrite compatibility for v0.1.8; defer mandatory protocol: websocket migration to a breaking-allowed train with handling for existing stored policies.
  • PR: fix(network): align REST and WebSocket endpoint handling #5336
Review details
  • Execution: complete
  • Candidate: PR #5336 (fd922893bf6c1acc33f92805ddd34787a5970d9b)
  • Baseline: e9ba8d9e3568f7ca1a6320914d0b17e0572fac2d (e9ba8d9e3568f7ca1a6320914d0b17e0572fac2d)
  • Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

@johntmyers johntmyers added area:runtime Sandbox execution, isolation, supervision, policy enforcement, networking, and provider lifecycle. test:e2e Requires end-to-end coverage labels Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Label test:e2e applied for 7b26195. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@johntmyers

Copy link
Copy Markdown
Contributor Author

gator-agent

PR Review Status

Reviewed the complete REST/WebSocket handling change and found no blocking findings. REST upgrades are refused across relay and forwarding paths, credential-rewrite validation is consistent across policies, provider profiles, and runtime configuration, and the docs explain the migration to explicit WebSocket endpoints.

Required checks are still running. I applied area:runtime for reviewer routing and test:e2e for policy and network coverage, then followed E2E Label Help's instructions to rerun the current-head Branch E2E Checks workflow; attempt 2 is running. Maintainer approval remains outstanding after CI completes.

Blocking findings: None.

Carried findings: None.

Gator metadata
  • Validation: Focused network-policy correction from a verified repository administrator.
  • Docs: Relevant Fern policy schema, network-rule migration guidance, provider documentation, and policy-generation skill updated.
  • Checks: DCO and vouch passed; Branch Checks pending; Helm Lint and Trivy Changes green. Area label applied to repair Review Readiness routing; its next result remains to be checked.
  • E2E: test:e2e applied; current-head Branch E2E Checks run 42244869447, attempt 2, in progress. No Windows/GPU-specific change requires another label.
  • Review validation: Independent code-only review; findings JSON validated. No local test execution by Gator.
  • Head SHA: 7b26195d4400ce8e32425ec545cce6bbacb9b9f4
  • Base SHA: e9ba8d9e3568f7ca1a6320914d0b17e0572fac2d
  • Merge base SHA: e9ba8d9e3568f7ca1a6320914d0b17e0572fac2d
  • Patch ID: 1a7bfba5e5ab1b8cc0c2f4c002596ec320bdd23e
  • Gator payload: 13
  • Review mode: initial
  • Previous reviewed SHA: none
  • Review budget exhausted: no
  • Maintainer decision required: no
  • Next state: gator:watch-pipeline

@johntmyers johntmyers added the gator:watch-pipeline Gator is monitoring PR CI/CD status label Nov 6, 2026

@russellb russellb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

I reviewed this prior to posting and it has since been updated to address the one finding I had.

@johntmyers johntmyers added gator:blocked Gator is blocked by process or repository gates and removed gator:watch-pipeline Gator is monitoring PR CI/CD status labels Nov 6, 2026

@purp purp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚢

@johntmyers
johntmyers added this pull request to the merge queue Nov 6, 2026
@johntmyers johntmyers added gator:merge-ready and removed gator:blocked Gator is blocked by process or repository gates labels Nov 6, 2026
Merged via the queue into main with commit 9321d16 Nov 6, 2026
397 of 407 checks passed
@johntmyers
johntmyers deleted the fix/rest-protocol-consistency/johntmyers branch November 6, 2026 17:57
@johntmyers

Copy link
Copy Markdown
Contributor Author

gator-agent

Monitoring Complete

Monitoring is complete because this PR has merged. The prior Gator review found no blocking findings, and the last active state was gator:merge-ready.

I removed the active gator:* label because there is nothing left for Gator to monitor on this PR.

Gator metadata
  • Head SHA: 7b26195d4400ce8e32425ec545cce6bbacb9b9f4
  • Gator payload: 13

cv added a commit to NVIDIA/NemoClaw that referenced this pull request Nov 16, 2026
Moves NemoClaw's OpenShell pins from v0.1.7 to v0.1.8: the native CLI,
the gateway, supervisor and sandbox images, the gateway chart, the
OpenShell crates, and the vendored SDK. Deployments built from this
revision run OpenShell v0.1.8, which no longer waits out a Kubernetes
sandbox pod's termination grace period on delete and renews
client-credentials tokens in the CLI.

## Problem

NemoClaw pins OpenShell v0.1.7 (`5d25211`). v0.1.8 (`f3d9765`, 80
commits later) is the latest release. Among its fixes:

- deleting a sandbox on Kubernetes no longer waits out the pod's
30-second termination grace period (ai-openshell/OpenShell#5279), which every
Kubernetes destroy and sandbox replacement paid (ai-openshell/OpenShell#5025);
- the CLI renews an expired client-credentials token by repeating the
grant (ai-openshell/OpenShell#4389, closing ai-openshell/OpenShell#4287), a limit
our Kubernetes guide and GPU autoscaling example documented;
- deleted sandboxes no longer leave memory behind on the gateway
(ai-openshell/OpenShell#5428).

## Change

- **Pins:** the Cargo revision of `openshell-core`, `openshell-sdk` and
`openshell-policy`, and of the vendored SDK's `openshell-core` and
`openshell-crypto`; in `versions.json`, the version, revision, Linux CLI
archives, gateway, supervisor and sandbox images, and gateway chart; the
managed-gateway reference, a contract test and the gateway contract
module, which carry the gateway digest. The CLI checksums come from the
release's `openshell-checksums-sha256.txt` and match the downloaded
archives; the image and chart digests come from GHCR's `0.1.8` tags, and
GHCR's `0.1.7` tags match the previous pins.
- **Vendored SDK:** upstream changed only `src/client.rs` (a new
`auth_interceptor()` accessor), copied unchanged. The SDK manifest still
enables core telemetry, so the manifest patch stays; `NOTICE.md` records
the refresh. `Cargo.lock` changes only the five OpenShell crates'
revision.
- **SDK integration limits:** the table in
`internal-docs/design/architecture.md` still holds at v0.1.8 and now
cites that revision.
- **`websocket_credential_rewrite`:** OpenShell now rejects it on a
`rest` endpoint (ai-openshell/OpenShell#5336), through the `openshell-policy`
validation NemoClaw already runs on explicit policies. The field's
description says it needs `protocol: websocket`; the JSON Schema,
configuration reference and provider schema snapshot are regenerated.
- **Live / Kind budgets:** with v0.1.8 the two kind cases take about
35-40 s less each: 98-108 s and 62-70 s on this pull request's runs,
against 138-143 s and 94-100 s on recent `v1` runs. The per-test budget
and the `live-kind` slow-timeout drop from 180 s to 140 s, and the
step's wall budget from 300 s to 240 s.
- **Limits removed:** the Kubernetes guide and the GPU autoscaling
example no longer tell readers to sign in again when a
client-credentials token expires; they say to keep
`OPENSHELL_OIDC_CLIENT_SECRET` set.
- **Release notes** list the new pin, the fixes above, and the policy
changes that can stop an existing explicit policy:
`websocket_credential_rewrite` on `rest`, filesystem paths through a
symlink in a writable workspace (ai-openshell/OpenShell#5278), and IPv4-mapped
IPv6 endpoint hosts (ai-openshell/OpenShell#5101).

Resolves #13959.

## Upgrade notes checked

- **Static credential placement** (ai-openshell/OpenShell#4886): every profile
NemoClaw registers (inference, `api` integrations, Brave and Tavily
search) already declares `auth_style` and `header_name`, including
Anthropic's `x-api-key` and Brave's `X-Subscription-Token`, so their
credentials keep resolving.
- **OCSF attribution** (ai-openshell/OpenShell#5048): NemoClaw reads no OCSF
records.
- **SDK moves:** the Python and Go SDK changes don't apply; NemoClaw
uses the Rust SDK.
- **Gateway interceptors** (ai-openshell/OpenShell#5173): NemoClaw registers
none.
- **Chart:** v0.1.8 adds optional `metrics.tls`, `priorityClassName` and
`topologySpreadConstraints`; NemoClaw sets none of them.
ai-openshell/OpenShell#5478, which would replace the `metricsPort: 0` setting,
isn't in v0.1.8, so that setting stays.
- **User guide:** apart from the lines above, `docs/` still describes
NemoClaw `637bdb6` and its OpenShell 0.1.7 links; moving them belongs to
the next guide refresh.

---------

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
ilanh added a commit to ilanh/openshell-annotate that referenced this pull request Nov 16, 2026
…pped hosts (#97)

Closes #95

Lint warnings for OpenShell v0.1.8 runtime changes that the policy
validator does not catch (split out of #80 to keep the bump PR #96
reviewable; independent of it).

## Change
- **WebSocket on `rest`** (ai-openshell/OpenShell#5336, REST endpoints answer
upgrades with 403): `warning[net]` for a `ws://`/`wss://` URL literal
whose host:port is declared only by an inspected `rest` egress
(`protocol=rest`, or `access=`/`allow=` with no protocol). Not reported
when any `protocol=websocket` (or other non-rest) egress also covers it,
for L4 passthrough egress, on comment-only lines, in excluded files, or
under a nearby `@openshell ignore kind=net`. Engine-independent (works
with `engine = "regex"` too).
- **IPv4-mapped hosts** (ai-openshell/OpenShell#5101, evaluated as IPv4): new
`validate::host_warning` (any `::ffff:` form, bracketed or hex) →
`warning[directive]` on egress directives, and a warning from
`check_document`, so `lint --policy`, `drift` and `generate` report it
in policy files.
- **Symlinked fs paths** (ai-openshell/OpenShell#5278): not detectable
statically (the symlink is in the image); README "Limits" says so and
what to do.

## Docs
README "Limits" bullet; CHANGELOG Unreleased. No CLI flags or directive
syntax changed, so `docs/CLI.md`/`docs/DIRECTIVES.md` are unchanged
(`gen-docs.sh --check` passes).

## Verified (Rust 1.94.1)
`scripts/ci-local.sh` passes (fmt, clippy `-D warnings`, all tests, docs
checks). New tests: `lint_warns_about_websocket_urls_on_rest_egress`,
`lint_policy_warns_about_ipv4_mapped_hosts`, unit
`ipv4_mapped_hosts_warn`. `examples/polyglot` still lints clean with
`--deny-warnings`, so `actions-selftest` is unaffected.

Not merging; needs Ilan's approval.

Signed-off-by: Ilan Harel <ilanharel@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:runtime Sandbox execution, isolation, supervision, policy enforcement, networking, and provider lifecycle. test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants