Skip to content

feat(runtime): join supervisors over the control connection - #5791

Draft
drew wants to merge 6 commits into
mainfrom
feat/5790-token-bootstrap/drew
Draft

drew wants to merge 6 commits into
mainfrom
feat/5790-token-bootstrap/drew

Conversation

@drew

@drew drew commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Kubernetes, Docker, Podman, and VM sandboxes now join their supervisor using a short-lived Join token and receive configuration over the existing control connection. The token bundles the supervisor CA digest, route, and generation-bound JWT. After TLS verification and durable single-use redemption, the supervisor returns configuration and a renewable runtime-only credential; the same TLS stream then carries the Sandbox Protocol.

Remove the separate bootstrap listener, port 5502, paired Service/NetworkPolicy, bootstrap routes, and sandbox-side TLS certificate staging. Kubernetes uses two Secret-backed environment inputs: Join token and supervisor address. Local drivers stage only a consumable credential envelope in existing protected storage so immutable containers can receive fresh credentials on restart. Workload configuration and TLS private keys stay supervisor-side.

Security and lifecycle

  • Pin the presented root CA and retain normal WebPKI chain, name, validity, and handshake-signature checks before sending the Join bearer.
  • Restrict Join JWTs to one sandbox generation and epoch with a distinct audience/component/type and 15-minute lifetime. Reuse the existing JWT implementation.
  • Atomically record redemption through a supervisor-authenticated gateway RPC. Replays stay rejected across supervisor restart and shared-pool failover. Losing the response after redemption requires a fresh generation.
  • Reconnect locally using runtime-only JWTs with the existing one-hour maximum. Refresh unused spares every 30 minutes, retaining active control connections.
  • Strip protected launch inputs from workload children. Rotate launch material on stop/start and preserve workspace state.

The initial exchange retains the existing private JSON configuration model. The canonical protobuf/SDK work in the stack ending at #5519 remains applicable. Deploy matching binaries/images together; local sandboxes created with the earlier file-based runtime contract require recreation.

Testing

  • Live Docker Join-token E2E: create, exec, configuration/credential isolation, stop/start, argv, cwd, environment, persisted workspace.
  • Live Podman Join-token E2E with the same parity assertions.
  • Live VM Join-token E2E plus focused CLI create/exec/list/delete conformance.
  • Existing Docker, Kubernetes, and Podman unit suites (743 passed; two pre-existing Docker tests ignored). No new unit tests.
  • Public/durable schema inventory check; only two public messages and one RPC are added.
  • TypeScript SDK CI and crypto dependency policy checks.
  • Common E2E target compiles and passes Clippy for all four driver feature sets. Kubernetes coverage includes wrong-pin, Join-as-session, and spent-token rejection against a live supervisor.
  • Live Kubernetes E2E: blocked before OpenShell deployment because host inotify exhaustion prevents containerd CRI startup (failed to create fsnotify watcher: too many open files). Temporary cluster removed.
  • Workspace Clippy, sandbox startup logging smoke test, formatting, lockfile alignment, Markdown/proto/Helm checks, and repository commit hooks.

Release Notes

Sandboxes use Join tokens to receive configuration and establish their control session on one TLS connection. Roll out matching gateway, supervisor, sandbox, and external driver binaries/images. Recreate local sandboxes that use the earlier file-based runtime contract.

Closes #5790. Related to #5456 and #5519.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Nov 20, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the state:accepted A maintainer decided OpenShell should pursue this issue label Nov 20, 2026
@github-actions

Copy link
Copy Markdown

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@drew drew changed the title feat(kubernetes): prototype token-based boundary bootstrap feat(kubernetes): fetch boundary configuration from supervisors Nov 20, 2026
drew added 4 commits November 19, 2026 21:33
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@drew drew changed the title feat(kubernetes): fetch boundary configuration from supervisors feat(runtime): join supervisors over the control connection Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state:accepted A maintainer decided OpenShell should pursue this issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(runtime): bootstrap sandbox drivers with Join tokens

1 participant