Repository navigation
ci(bench): serialise the Pages deployment so overlapping publishers cannot lose the deploy - #2008
Merged
Merged
Conversation
…roup #2005 gave the nightly its own concurrency group, so a nightly and a push run can publish at the same time. #2007 made that safe on the benchmark-stats branch, but the deploy-pages job still has no serialisation: Pages allows one deployment in progress per environment, so the loser fails with "Deployment failed, try again later" (actions/deploy-pages#406, #418) and main goes red for a reason unrelated to the measurement. Add a job-level concurrency group (github-pages, cancel-in-progress: false), matching GitHub's own Pages template -- job level so `publish` keeps pushing history while only the deployment queues -- and enforce it: a new check_pages_rules in ci/check_workflow_concurrency.py rejects a deploy-pages job without a shared group, with a per-run group, or with cancel-in-progress: true.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
benchmark-build-comparison.ymldeliberately lets two runs publish at once:#2005 gave the nightly its own
concurrency:group so a merge could no longerevict it, which means a nightly and a push run now reach the
deploy-pagesjob concurrently. #2007 made overlapping publishers safe on the
benchmark-statsbranch (re-read per attempt +--force-with-lease+ historyunion), but the deployment itself has no such protection:
docs, "Using concurrency": "so that an environment has a maximum of one
deployment in progress at a time"). The run that loses that race fails the
actions/deploy-pagesstep withDeployment failed, try again later(Intermittent Deploy Failures starting 2026-01-13 actions/deploy-pages#406, chore(deps): bump zccache pin 1.11.14 -> 1.11.15 #418) — a red bench run on
mainfor a reason thathas nothing to do with the measurement.
cancel-in-progress: trueis not a fix: cancelling mid-deployment can leavethe environment stuck and poisons the next run (same issues).
publishjob still claimed "a single assemblerstays authoritative -- two publishers would clobber", which stopped being true
when ci(bench): give the nightly its own concurrency slot, union history at publish #2005 landed.
GitHub's own Pages template serialises with
concurrency: {group: github-pages, cancel-in-progress: false}. This PR appliesthat to the deployment job and makes it structural instead of tribal knowledge.
What
.github/workflows/benchmark-build-comparison.yml— job-levelconcurrency: {group: github-pages, cancel-in-progress: false}ondeploy-pages. Job-level (not workflow-level) on purpose:publishkeepsrunning and keeps pushing history, only the deployment queues. A
workflow-level group would also have to be shared with the measurement job,
which is exactly what ci(bench): give the nightly its own concurrency slot, union history at publish #2005 fixed.
ci/check_workflow_concurrency.py— newcheck_pages_rules: any job thatruns
actions/deploy-pagesmust declare a job-levelconcurrency:whosegroup:is a constant (a group containinggithub.run_id/run_number/sha/ref/head_refserialises nothing) and must not setcancel-in-progress: true. A job that only uploads the artifact(
actions/upload-pages-artifact) is not a deployment and is ignored.publishcorrected to describe the lease + the deployserialisation.
Proof
TDD, red first — the rule did not exist:
Then, with the rule in place and the workflow unpatched, the real repo failed
the gate — this is the defect, caught on the shipped workflow:
After patching the workflow:
Full local gates (the exact commands
ci-workflow-drift.ymlruns):python -m unittest ci.test_workflow_concurrency— 16 tests OK(13 pre-existing + 5 new cases: no block / shared group accepted / per-run
group rejected / cancel-true rejected / upload-only job ignored, minus the
repo-wide test that now covers the real file).
ruff==0.12.12 format --check+ruff checkon both files: clean.ci/check_python_complexity.py:All checks passed!Parsed back from the workflow itself:
Not done on purpose
trigger (owner cost call), and no change to
publish(a job-level group therecould cancel a pending publish and lose that run's row — the opposite of what
ci(bench): overlapping publishers can silently revert each other's history row #2006/ci(bench): publish under a lease so overlapping runs cannot revert history #2007 protect).