Repository navigation
Description
User Story
I maintain OpenShell releases. Users install OpenShell with curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh. I need Release Qualification to run that installer against every pre-release, both as a fresh install and as an upgrade from the previous stable, so a broken install path blocks promotion to stable.
Description
Release Qualification installs candidate .deb and .rpm files directly through the package manager (#3454). The upgrade work in #2975 (#3521, #4202) also uses the package manager. Neither runs install.sh.
The only end-to-end install.sh check is Release Canary. It:
- installs the rolling
devbuild, not the pre-release or stable; - runs after Release Dev publishes, so it cannot block anything;
- notifies no one. Its snap lane has been red since 2026-10-29 while v0.1.7, v0.1.8, and v0.1.9 shipped.
install.sh regressions have reached users before; for example, #4412 was an install.sh/brew upgrade failure.
The canary also covers more than a basic install. Removing it must not lose:
- the Helm chart published to GHCR, installed on kind with the CLI connected to it;
- snap behavior:
install.shuses system Docker and does not install the Docker snap, thedockerinterface is connected, the user gateway is enabled only for the installing user and not for a second user (fix(snap): only enable gateway user service for one user #4615), the system gateway stays off, mTLS user authentication is on, andopenshell-proverworks; install.shrefusing to install on snap hosts with no Docker or with only the Docker snap;- installing from the published channels (GitHub release assets, Snap Store, GHCR), which catches broken publishing that artifact-based qualification cannot.
The fix is to add install.sh scenarios to Release Qualification, add a post-publish smoke test, and remove Release Canary once both are required.
Context
- Parent: feat: 0.1.0 release pipeline #2860 (RFC 0014 release pipeline). See upgrade tests.
- Builds on ci(qualification): add reusable N-to-N+1 upgrade testsuite #2975 (N-to-N+1 upgrade test suite) and test(continuity): add suites for gateway restart and package upgrade #3521 / test(continuity): upgrade rpm packages on Podman #4202 (DEB/RPM upgrade lanes). This issue covers driving those flows through
install.shand adding the missing package types. It does not replace them. - Upgrade tests alone do not cover a fresh install of the candidate. The source install runs N's first-run path (PKI generation, default config, empty state), not N+1's.
- Discussion: https://nvidia.slack.com/archives/C0AR6QP0CKH/p1795046862720489
Definition of Done
- Release Qualification runs
install.shfrom the candidate commit withOPENSHELL_VERSION=<pre-release tag>on a clean host, then creates, executes in, and deletes a sandbox. - Release Qualification runs an upgrade scenario:
install.shinstalls the previous stable, creates a gateway and sandbox, theninstall.shupgrades to the pre-release. It verifies that existing state survived and that a new sandbox works. - Both scenarios cover deb (Ubuntu), rpm (Fedora), snap (Ubuntu), Homebrew (macOS), and the Helm chart (kind). Helm upgrades use
helm upgradefrom the previous stable chart, and the CLI comes frominstall.sh. - Homebrew on GitHub-hosted macOS runners checks
openshell statusonly, because the runners do not expose Hypervisor.framework. Sandbox coverage on macOS needs a runner with virtualization or is explicitly documented as not covered. - The snap scenarios keep the canary's snap checks: no Docker snap installed,
dockerinterface connected, user gateway enabled for the installing user only (not a second user), system gateway disabled, mTLS user authentication enabled, and anopenshell-prover checkrun. - Qualification checks that
install.shfails with a clear error on a snap host with no Docker and on a host with only the Docker snap. - deb and rpm scenarios also run on arm64.
- Both scenarios are required for qualification, so a failure blocks stable promotion.
- Failures keep installer output and gateway service logs.
- Release (
release.yml) ends with a smoke test that installs the publishedvX.Y.Zthroughinstall.shfrom the real channels (GitHub release, Snap Storelatest/stable, Homebrew, GHCR chart) and creates a sandbox. Stable artifacts are rebuilt after qualification (chore(release): evaluate promoting qualified pre-release artifacts instead of rebuilding for stable #3632 closed as not planned), so this is the only check on what users download. A failure fails the workflow and notifies the release channel. - Release Canary (
.github/workflows/release-canary.yml) is removed, and thetest-release-canaryandrelease-captainskills are updated.
+++ Agent notes
- Canary snap lane root cause, if it needs to stay green until removal: GitHub runner provisioning writes
XDG_RUNTIME_DIR=/run/user/1001to/etc/environment(XDG_RUNTIME_DIR is set system-wide in /etc/environment, breaking sessions of other users actions/runner-images#14649). The canary's second user inherits it, and on systemd 257+ (Ubuntu 26.04) snapd's per-usersystemctl --user daemon-reloadfails. Workaround:sudo sed -i '/^XDG_RUNTIME_DIR=/d' /etc/environmentbefore creating the user. - Existing canary jobs to port, by line in
.github/workflows/release-canary.yml: macOS Homebrew L22, Ubuntu DEB L56, Fedora RPM L94 (systemd container with Podman), snap with system Docker L194, snap Docker preflight L316, Kubernetes Helm on kind L378 (installs the Agent Sandbox controller viae2e/support/install-agent-sandbox.sh). - Local repro observations (arm64 containers, v0.1.9):
install.shusescurl -fLs --retry 3, which hides errors and does not retry mid-transfer resets. The Docker driver pulls the supervisor image before the gateway listens (ensure_runtime_image(..., "supervisor")incrates/openshell-driver-docker/src/lib.rs), so a slow pull exceeds the installer's 30s listener wait. Track separately if confirmed.
+++
Activity
[-]ci(qualification): qualify install.sh fresh installs and upgrades, then retire Release Canary[/-][+]ci(release): qualify install.sh installs and upgrades, smoke-test published releases, then retire Release Canary[/+]