Repository navigation
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
🌿 Preview your docs: https://nvidia-preview-pr-4202.docs.buildwithfern.com/openshell |
dff9ea0 to
cb06c80
Compare
|
This pull request has had no activity for 14 days and is now marked stale. |
cb06c80 to
295e946
Compare
|
This pull request has had no activity for 14 days and is now marked stale. |
295e946 to
f2f756f
Compare
148cadd to
edb874b
Compare
f580d45 to
d3f14ae
Compare
edb874b to
c22cd2b
Compare
d3f14ae to
9a30af5
Compare
c22cd2b to
f044425
Compare
matthewgrossman
left a comment
There was a problem hiding this comment.
Approving. Note it has had no CI run on this head (stacked on #3521, continuity skipped) and the local runs used arm64 while CI is amd64, so the rpm download path is untested in CI.
The four unexplained SSH Permission denied flakes concern me now that continuity gates releases. Could we agree a plan (retry, or non-blocking until understood) before this merges?
Also, the rpm step waits up to 10s to detect the package's own restart. If there's a way to assert it from the unit's InvocationID instead of polling, I'd prefer that.
|
Label |
The base branch was changed.
I have a local commit to switch to ussing |
I would need to check this. |
|
Re the 10s wait in the rpm upgrade step: it already asserts from the unit's |
Add an `rpm-upgrade-source` installer that provisions a Fedora guest from the latest release's openshell and openshell-gateway packages and release-tagged images, and stages the candidate packages and images with the `upgrade` disruption that installs them. The scenarios stay unchanged and run under it like any other disruption. The upgrade step selects the candidate's runtime images before it installs, because the gateway package restarts the user service itself after an upgrade (%systemd_user_postun_with_restart). It restarts the service only if the package did not: a second restart right after the package's interrupts the new gateway while it brings its sandboxes back, which left the pre-upgrade sandbox in an error phase. It checks both installed packages against the candidates' versions and supports rootless Podman only. Teach the prepare-upgrade-source action the rpm package type, which downloads both RPMs and verifies them against the release's single checksum file, and add a fedora-podman-rootless row that runs with `upgrade-source`. Signed-off-by: Evan Lezar <elezar@nvidia.com>
Render the rpm upgrade step for the user that runs the gateway, which the packaged-gateway role detects: the login user under rootless Podman and root under rootful. Every action in the step, including loading the images into Podman storage, writing gateway.env, and talking to the user manager, runs as that user. The step no longer refuses rootful Podman, and checks that the service is loaded for the gateway user through `show` instead of `cat`, which fails for root. Add fedora-podman-rootful rows to the continuity matrix: gateway-restart on the rpm installer, and upgrade on rpm-upgrade-source with the other upgrade rows. Signed-off-by: Evan Lezar <elezar@nvidia.com>
sshpass sometimes answered the password prompt with a password sshd rejected, which failed about 3% of logins on a loaded host with 'Permission denied'. tmachine opens a new connection for every command, so a run made hundreds of logins and hit the flake often. Pass the password through SSH_ASKPASS instead. It failed none of 180 logins in the same test. Signed-off-by: Evan Lezar <elezar@nvidia.com>
The baseline is the latest release when the run starts, so the same candidate can pass one day and fail the next. Write the tag to the run summary and a notice annotation, in the select job and in each upgrade row, so a result says which release it upgraded from. Signed-off-by: Evan Lezar <elezar@nvidia.com>
Record that the wait for the package's restart should be measured and, if the restart is already visible when dnf returns, replaced with a single InvocationID read. Signed-off-by: Evan Lezar <elezar@nvidia.com>
f044425 to
dbcc79f
Compare
Summary
Add the RPM variant of the
upgradedisruption from #3521, and cover Podman in the continuity matrix. A Fedora guest starts on the latest published release'sopenshellandopenshell-gatewaypackages, upgrades to the candidate, and the continuity scenarios check that a sandbox created before the upgrade still works afterwards. The upgrade runs under rootless and rootful Podman, andgateway-restartruns under rootful.Stacked on #3521, which holds the continuity family, the runner's
--disruptionsupport, the Debian upgrade, and the CI wiring. Its base stayscodex/ubuntu-deb-upgrade-testuntil that merges.Related Issue
Refs #2975 — reusable N-to-N+1 upgrade qualification.
Changes
rpm-upgrade-sourceinstaller. It provisions the guest from the latest release's two RPMs and its release-tagged images, and stages the candidate RPMs and images with theupgradedisruption, asdeb-upgrade-sourcedoes for Debian.dnf, requires both installed versions to equal the candidates', and runs the gateway on the candidate's images.prepare-upgrade-sourceaction therpmpackage type. It downloads both RPMs and verifies each against the release'sopenshell-checksums-sha256.txt.upgradeonfedora-podman-rootlessandfedora-podman-rootful, andgateway-restartonfedora-podman-rootful. Like the other continuity rows, they run in the release workflows. Document them inTESTING.mdandCI.md.Release Notes
Testing
Ran on macOS with HVF on Fedora 44 (arm64) guests, with the baseline v0.1.9 and the
devbuild as the candidate. These runs predate rebasing onto #3521's current head. The files this PR adds or changes are unchanged by the rebase, but the newermainalso changed tmachine's own Rust sources, so the guest runs have not been repeated on this head.continuitywithupgradeonrpm-upgrade-source: passes onfedora-podman-rootlessand onfedora-podman-rootful. The first rootless run failed because the step restarted the gateway twice, as described above, and the first rootful run failed on a check that returned non-zero for root.continuitywithgateway-restarton therpminstaller: passes on bothfedora-podman-rootlessandfedora-podman-rootful.shellcheckon the rendered step for both users, and on the action's download script;actionlintone2e-suites.yml; row selection gives 30 rows withoutupgrade-sourceand 33 with it, with unique artifact names.Not verified:
The new CI jobs have not run on this head. The
rpmbranch ofprepare-upgrade-sourcehas only been read and linted: it runs in CI on amd64, and the local runs used the arm64 packages.Four local guest runs failed with an SSH
Permission deniedon a guest tmachine had already reached, at points unrelated to this change. I haven't found the cause. Continuity rows now gate the release workflows, so a recurrence in CI could fail a run for reasons unrelated to OpenShell.The pre-commit hooks were skipped:
rust:lintcrashes with a clippy internal compiler error inopenshell-supervisor-network, which this PR doesn't touch.Checks appropriate to the affected code and behavior pass
Unit tests added/updated (if applicable)
E2E tests added/updated (if applicable)
Checklist