Skip to content

feat(helm): separate metrics enablement from listener port - #5478

Merged
TaylorMutch merged 2 commits into
mainfrom
feat/5474-separate-metrics-enablement/TaylorMutch
Nov 11, 2026
Merged

TaylorMutch merged 2 commits into
mainfrom
feat/5474-separate-metrics-enablement/TaylorMutch

Conversation

@TaylorMutch

@TaylorMutch TaylorMutch commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Use metrics.enabled to enable or disable the gateway's dedicated Prometheus endpoint independently of its port. Metrics remain enabled on port 9090 by default; disabling the endpoint retains the configured port and omits the runtime listener, container port, and Service port.

Related Issue

Closes #5474.

Changes

  • Add boolean metrics.enabled, defaulting to true, and use it consistently across runtime and Kubernetes resources.
  • Keep service.metricsPort between 1 and 65535; check listener collisions only while metrics are enabled.
  • Reject legacy metricsPort: 0 and false values with instructions to set metrics.enabled=false and retain a valid port. Requesting metrics TLS while disabled also fails rendering.
  • Cover default, custom, disabled, TLS, and migration configurations with chart tests and Rust-loader validation.
  • Update the generated chart README, published gateway configuration docs, and related operating skills.

Testing

  • mise run helm:test: 470 gateway and 7 workspace chart tests, generated gateway TOML validation through the Rust loader, and resource coherence checks
  • Helm lint and template rendering across maintained CI/dev overlays
  • Generated Helm README consistency
  • Fern documentation and navigation validation
  • Markdown lint, ShellCheck, and git diff whitespace checks
  • Live Kubernetes deployment smoke test

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • User-facing configuration documentation and relevant skills updated

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Nov 10, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown

@TaylorMutch
TaylorMutch marked this pull request as ready for review November 10, 2026 23:53
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
@github-actions

Copy link
Copy Markdown

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. helm — No explicit risk acceptance found

service.metricsPort: 0 previously disabled metrics but now fails chart validation, even with metrics.enabled=false.

  • Impact: Existing Helm values containing service.metricsPort: 0 block rendering and upgrades until operators set metrics.enabled=false and replace the zero port with a valid port.
  • Action: Retain service.metricsPort: 0 as a backward-compatible disabled-metrics alias on this no-break train rather than requiring existing installations to migrate.
  • PR: feat(helm): separate metrics enablement from listener port #5478
Review details
  • Execution: complete
  • Candidate: PR #5478 (786cb907fbfd415428dc885cf71603124de882c3)
  • Baseline: 4ad3ce9db2700da6a3d32f8179f68062b7aaa6c3 (4ad3ce9db2700da6a3d32f8179f68062b7aaa6c3)
  • Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

@TaylorMutch

Copy link
Copy Markdown
Contributor Author

This fixes unclear behavior in the metrics port enablement so compatibility is not a concern.

@TaylorMutch
TaylorMutch added this pull request to the merge queue Nov 11, 2026
Merged via the queue into main with commit 788bbdc Nov 11, 2026
92 checks passed
@TaylorMutch
TaylorMutch deleted the feat/5474-separate-metrics-enablement/TaylorMutch branch November 11, 2026 00:38
jgarciao pushed a commit to jgarciao/OpenShell that referenced this pull request Nov 11, 2026
…ell#5478)

* feat(helm): separate metrics enablement from listener port

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

* docs(helm): simplify metrics enablement guidance

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>

---------

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
cv added a commit to NVIDIA/NemoClaw that referenced this pull request Nov 11, 2026
…n bump

v1's broader #14316 row already covers service URLs on a published
gateway. The chart values note that ai-openshell/openshell#5478 replaces
metricsPort 0 with metrics.enabled false.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
cv added a commit to NVIDIA/NemoClaw that referenced this pull request Nov 16, 2026
Moves NemoClaw's OpenShell pins from v0.1.7 to v0.1.8: the native CLI,
the gateway, supervisor and sandbox images, the gateway chart, the
OpenShell crates, and the vendored SDK. Deployments built from this
revision run OpenShell v0.1.8, which no longer waits out a Kubernetes
sandbox pod's termination grace period on delete and renews
client-credentials tokens in the CLI.

## Problem

NemoClaw pins OpenShell v0.1.7 (`5d25211`). v0.1.8 (`f3d9765`, 80
commits later) is the latest release. Among its fixes:

- deleting a sandbox on Kubernetes no longer waits out the pod's
30-second termination grace period (ai-openshell/OpenShell#5279), which every
Kubernetes destroy and sandbox replacement paid (ai-openshell/OpenShell#5025);
- the CLI renews an expired client-credentials token by repeating the
grant (ai-openshell/OpenShell#4389, closing ai-openshell/OpenShell#4287), a limit
our Kubernetes guide and GPU autoscaling example documented;
- deleted sandboxes no longer leave memory behind on the gateway
(ai-openshell/OpenShell#5428).

## Change

- **Pins:** the Cargo revision of `openshell-core`, `openshell-sdk` and
`openshell-policy`, and of the vendored SDK's `openshell-core` and
`openshell-crypto`; in `versions.json`, the version, revision, Linux CLI
archives, gateway, supervisor and sandbox images, and gateway chart; the
managed-gateway reference, a contract test and the gateway contract
module, which carry the gateway digest. The CLI checksums come from the
release's `openshell-checksums-sha256.txt` and match the downloaded
archives; the image and chart digests come from GHCR's `0.1.8` tags, and
GHCR's `0.1.7` tags match the previous pins.
- **Vendored SDK:** upstream changed only `src/client.rs` (a new
`auth_interceptor()` accessor), copied unchanged. The SDK manifest still
enables core telemetry, so the manifest patch stays; `NOTICE.md` records
the refresh. `Cargo.lock` changes only the five OpenShell crates'
revision.
- **SDK integration limits:** the table in
`internal-docs/design/architecture.md` still holds at v0.1.8 and now
cites that revision.
- **`websocket_credential_rewrite`:** OpenShell now rejects it on a
`rest` endpoint (ai-openshell/OpenShell#5336), through the `openshell-policy`
validation NemoClaw already runs on explicit policies. The field's
description says it needs `protocol: websocket`; the JSON Schema,
configuration reference and provider schema snapshot are regenerated.
- **Live / Kind budgets:** with v0.1.8 the two kind cases take about
35-40 s less each: 98-108 s and 62-70 s on this pull request's runs,
against 138-143 s and 94-100 s on recent `v1` runs. The per-test budget
and the `live-kind` slow-timeout drop from 180 s to 140 s, and the
step's wall budget from 300 s to 240 s.
- **Limits removed:** the Kubernetes guide and the GPU autoscaling
example no longer tell readers to sign in again when a
client-credentials token expires; they say to keep
`OPENSHELL_OIDC_CLIENT_SECRET` set.
- **Release notes** list the new pin, the fixes above, and the policy
changes that can stop an existing explicit policy:
`websocket_credential_rewrite` on `rest`, filesystem paths through a
symlink in a writable workspace (ai-openshell/OpenShell#5278), and IPv4-mapped
IPv6 endpoint hosts (ai-openshell/OpenShell#5101).

Resolves #13959.

## Upgrade notes checked

- **Static credential placement** (ai-openshell/OpenShell#4886): every profile
NemoClaw registers (inference, `api` integrations, Brave and Tavily
search) already declares `auth_style` and `header_name`, including
Anthropic's `x-api-key` and Brave's `X-Subscription-Token`, so their
credentials keep resolving.
- **OCSF attribution** (ai-openshell/OpenShell#5048): NemoClaw reads no OCSF
records.
- **SDK moves:** the Python and Go SDK changes don't apply; NemoClaw
uses the Rust SDK.
- **Gateway interceptors** (ai-openshell/OpenShell#5173): NemoClaw registers
none.
- **Chart:** v0.1.8 adds optional `metrics.tls`, `priorityClassName` and
`topologySpreadConstraints`; NemoClaw sets none of them.
ai-openshell/OpenShell#5478, which would replace the `metricsPort: 0` setting,
isn't in v0.1.8, so that setting stays.
- **User guide:** apart from the lines above, `docs/` still describes
NemoClaw `637bdb6` and its OpenShell 0.1.7 links; moving them belongs to
the next guide refresh.

---------

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
cv added a commit to NVIDIA/NemoClaw that referenced this pull request Nov 18, 2026
Moves NemoClaw's OpenShell pins from v0.1.8 to v0.1.9: the native CLI,
the gateway, supervisor and sandbox images, the gateway chart, the
OpenShell crates, and the vendored SDK. A published managed Kubernetes
gateway now disables the chart's metrics with `metrics.enabled: false`,
which v0.1.9 requires, and the live middleware fixture moves to
OpenShell's HTTP hooks. Deployments built from this revision run
OpenShell v0.1.9, whose chart applies `caConfigMap`'s CA only to the
OIDC issuer.

## Problem

NemoClaw pins OpenShell v0.1.8 (`f3d9765`). v0.1.9 (`ce58cf8`, 51
commits later) is the latest release, and it changes three things
NemoClaw depends on:

- The chart refuses `service.metricsPort: 0` (NVIDIA/OpenShell#5478),
which NemoClaw sets on a gateway published through `expose.service`; the
chart's exposed contract scenario fails to render with
`service.metricsPort must be an integer between 1 and 65535; to disable
metrics, set metrics.enabled=false`.
- `server.oidc.caConfigMapName` no longer sets a process-wide
`SSL_CERT_FILE` on the gateway; the chart renders
`[openshell.gateway.oidc] ca_bundle`, so the CA applies only to OIDC
discovery and JWKS requests (NVIDIA/OpenShell#3673).
- Middleware selectors with a port fail validation, and the legacy
`EvaluateHttpRequest` and `HttpResponsePreReturn` hooks are deprecated,
to be removed in 0.2.0 (NVIDIA/OpenShell#4359).

Part of #13829: it settles question 2 of [cv's
answers](#13829 (comment)),
keeping `caConfigMap` as a typed field under
`target.kubernetes.authentication`. With v0.1.8 the chart turned that
ConfigMap into the gateway's process-wide `SSL_CERT_FILE`, so a field
named for the issuer's CA also set the trust of the gateway's other
HTTPS clients; with v0.1.9 it affects only the issuer, which is what a
typed authentication field says. The rest of #13829 (pull Secrets,
Docker and Podman ports, the `openshift` target) doesn't depend on this
upgrade, and the OpenShift UID nulls stay until NVIDIA/OpenShell#5483.

## Change

- **Pins:** the Cargo revision of `openshell-core`, `openshell-sdk` and
`openshell-policy`, and of the vendored SDK's `openshell-core` and
`openshell-crypto`; in `versions.json`, the version, revision, Linux CLI
archives, gateway, supervisor and sandbox images, and gateway chart; the
managed-gateway reference, a contract test and the gateway contract
module, which carry the gateway digest. The CLI checksums come from the
release's `openshell-checksums-sha256.txt`; the image and chart digests
come from GHCR's `0.1.9` tags, and GHCR's `0.1.8` tags match the
previous pins. `Cargo.lock` changes only the five OpenShell crates'
revision.
- **Vendored SDK:** upstream changed `src/client.rs` and `src/raw.rs` (a
raw `ServiceDiscovery` client), `src/transport.rs` and
`tests/client_mock.rs`, copied unchanged. The SDK manifest still enables
core telemetry, so the manifest patch stays; `NOTICE.md` records the
refresh. The SDK integration limits in
`internal-docs/design/architecture.md` still hold at v0.1.9 and cite
that revision.
- **Published gateway metrics:** `expose.service` sets the chart's
`metrics.enabled: false` instead of `service.metricsPort: 0`, so the
NodePort Service still publishes only the gRPC port and the gateway
serves no plaintext metrics. This deletes the `metricsPort` workaround
and its comment, which #14313 left for this upgrade. The chart
contract's exposed scenario failed with the v0.1.9 chart before the
change and passes after it.
- **Middleware:** the `docker-dummy` live case's middleware fixture
implements OpenShell's `HttpHooks` service and requires the
`openshell.supervisor-middleware.http-hooks` capability, deciding at
preflight without the body; it answers the legacy `EvaluateHttpRequest`
with `UNIMPLEMENTED`. The case's registration and assertions are
unchanged. A middleware selector with a port is a new case in the SDK's
OpenShell-validation test, which failed against v0.1.8 and passes
against v0.1.9; the selector's schema description and the sandbox
network guide say hosts take no port.
- **Docs:** OpenShell links move to the v0.1.9 revision; each path and
anchor exists there, and the three proxy line ranges in the sandbox
network guide move 71 lines down to the same text. The Kubernetes guide
says the gateway trusts `caConfigMap`'s CA only for the issuer.
- **Release notes** list the new pin and the v0.1.9 changes users see,
with two breaking changes: a middleware selector with a port fails plan,
and a DNS-glob endpoint no longer admits a private IP literal on its own
(NVIDIA/OpenShell#5522).

## Upgrade notes checked

- **Supervisor shutdown** (NVIDIA/OpenShell#5342): a dedicated
supervisor now stops within 8 s and Docker waits 10 s, so stops no
longer end in `SIGKILL`, and a connection that drops during shutdown is
no longer recovered. That can shorten the 13 s between stop and start in
#13953's failure, but the failure itself, a restarted supervisor finding
its sandbox session terminal, isn't addressed, NVIDIA/OpenShell#4910 is
still open, and #13953 didn't reproduce in 260 runs; #13953 stays open.
- **OpenShift UIDs:** NVIDIA/OpenShell#5483 is still open and not in
v0.1.9, so `openshift` keeps nulling the chart's UIDs (#13828).
- **OCSF** (NVIDIA/OpenShell#5606): NemoClaw reads no OCSF records; the
release notes tell log consumers how to select denials now.
- **Go SDK move, mutation admission limits, startup windows, service
discovery:** NemoClaw uses the Rust SDK and OpenShell's defaults;
authored gateway configuration can set the new keys.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(helm): separate metrics enablement from the listener port

2 participants