Skip to content

chore(ci): align all builders on the ideal setup-soldr configuration and cut per-push compute waste #170

Closed

Description

@zackees

chore(ci): align all builders on the ideal setup-soldr configuration and cut per-push compute waste

Context

Audit of every GitHub Actions workflow in this repo (2026-06-11, main HEAD 68ee09a) for setup-soldr coverage and configuration quality.

Coverage map (good news): every Rust-compiling workflow already uses zackees/setup-soldr@v0.9.62, including release builds:

Workflow Uses setup-soldr version: pin cache-key-suffix
_build.yml (also used by auto-release) yes unpinned (action default v0.7.51) ${{ inputs.runs-on }}
_lint.yml yes 0.7.53 (known-broken, see #158) ${{ inputs.runs-on }}
_unit-test.yml yes 0.7.53 ${{ inputs.runs-on }}
_integration-test.yml yes 0.7.53 ${{ inputs.runs-on }}
linux-x86-dwarf-smoke.yml yes 0.7.53 n/a
24 thin wrapper workflows inherit via workflow_call — —
auto-release.yml inherits via _build.yml — —

So the problem is not missing adoption — it is config drift and architectural waste.

Measured waste: one push to main triggers 26 workflow runs totaling ~206.8 minutes of runner compute. Slowest jobs for 68ee09a:

  • Windows ARM Build: 18.7 min
  • macOS x86 Lint: 17.5 min
  • Windows x86 Lint: 16.7 min
  • Windows x86 Build: 16.0 min
  • macOS ARM Build: 15.0 min
  • Windows ARM Lint: 14.8 min

Findings

  1. Split-brain soldr versions split the cache. _build.yml is unpinned (resolves to action default v0.7.51 / known-good for the aarch64-pc-windows-msvc cross-compile), while _lint.yml, _unit-test.yml, _integration-test.yml, and linux-x86-dwarf-smoke.yml still pin version: 0.7.53 — the exact version PR ci: drop soldr v0.7.53 pin from _build.yml to unbreak Windows ARM Build #159 removed from _build.yml because zccache v1.11.20 breaks the Windows ARM cross-compile with silent rustc failures (ci: Windows ARM Build broken since soldr v0.7.53 pin — revert _build.yml to v0.7.51 #158). Different soldr/zccache versions across jobs mean lint/test/build never share warm cache entries, and four workflows run a known-bad version.
  2. cache-key-suffix is keyed on runs-on, not the rust target. _build.yml builds different --target triples on the same runner label (e.g. windows-2025 hosts both x86_64 and aarch64-pc-windows-msvc builds), so distinct target artifacts collide into / thrash one cache key. setup-soldr's README recommends a per-target suffix.
  3. No warm-then-fan-out. setup-soldr ships a reusable rust-ci.yml (warm one cache, fan out lint/test/build jobs that restore it). We instead run 26 independent runs that each cold-start their own cache namespace. Even without adopting the upstream reusable workflow wholesale, a per-platform warm job that lint/unit/integration/build depend on would collapse most of the duplicate compilation.
  4. auto-release.yml rebuilds all 6 platforms from scratch even though CI on the very same SHA already produced identical wheel + binary artifacts minutes earlier. Releasing could download the CI artifacts (or at minimum hit a fully warm cache, see finding 3) instead of paying ~80+ min of duplicate builds.
  5. dylint toolchain reinstalled every lint run. _lint.yml installs nightly-2026-03-26 + cargo-dylint/dylint-link 6.0.1 from source-ish paths each run — a large fixed cost on every platform (lint jobs are 14–17.5 min). These should be cached (e.g. prebuild-deps / tool cache) or prebuilt.
  6. "Windows ARM" unit-test and lint wrappers are name-only. windows-arm-unit-test.yml / windows-arm-lint.yml pass runs-on: windows-2025 (an x86 host), duplicating the Windows x86 jobs while adding zero ARM coverage. Either point them at a real ARM runner or delete them.

Proposal (upgrade plan)

Phase 1 — drift fixes (small, immediate):

  1. Remove the version: 0.7.53 pins from _lint.yml, _unit-test.yml, _integration-test.yml, linux-x86-dwarf-smoke.yml so all five reusable workflows resolve the same soldr version as _build.yml. Keep the explanatory comment about ci: Windows ARM Build broken since soldr v0.7.53 pin — revert _build.yml to v0.7.51 #158 in one place.
  2. Change cache-key-suffix to include the rust target: ${{ inputs.runs-on }}-${{ inputs.rust-target || 'native' }} in all four reusable workflows.
  3. Delete or re-host the name-only Windows ARM lint/unit-test wrappers.

Phase 2 — structural (bigger wins):
4. Introduce a per-platform warm job (or adopt setup-soldr's rust-ci.yml reusable workflow / cross-targets input) so lint, unit, integration, and build on a given platform share one warmed cache instead of four cold starts.
5. Make auto-release.yml consume the wheel/binary artifacts already produced by CI for the release SHA instead of rebuilding all 6 platforms; fall back to building only when no artifact exists for that SHA.
6. Cache the dylint nightly toolchain + cargo-dylint/dylint-link binaries in _lint.yml.

Acceptance criteria

  • No workflow pins soldr 0.7.53; all five reusable workflows resolve the same soldr/zccache version.
  • cache-key-suffix disambiguates rust targets sharing a runner label.
  • Name-only Windows ARM lint/unit-test wrappers are removed or run on real ARM hosts (decision recorded in the PR).
  • Lint/test/build jobs on the same platform demonstrably share cache (visible as cache hits in soldr output on a second run).
  • auto-release.yml no longer rebuilds platforms whose artifacts exist from CI on the same SHA.
  • Total per-push compute for a main push measurably drops from the ~207-min baseline (record before/after in the PR).
  • All checks green on main after each phase merges.

Decisions

  • Scope: this is a plan/tracking issue; implementation lands as 2 phased PRs (drift fixes first, structural second) — phase 1 is low-risk and unblocks cache sharing immediately.
  • Priority: P1 — per the maintainer, remote build wait time is "incredibly high and wasteful"; ~207 min compute per push with 18-min critical path.
  • Soldr version strategy: unpin everywhere rather than pin everywhere — matches _build.yml's current green config and PR ci: drop soldr v0.7.53 pin from _build.yml to unbreak Windows ARM Build #159 rationale; revisit pinning once soldr > 0.7.53 fixes ci: Windows ARM Build broken since soldr v0.7.53 pin — revert _build.yml to v0.7.51 #158.
  • Windows ARM test/lint wrappers: recommend deletion — they duplicate x86 coverage under a misleading name; real ARM coverage already exists via the aarch64-pc-windows-msvc cross-compile build.
  • Implementing agent: fable-5 with xhigh reasoning effort — per maintainer's standing instruction for CI work.

Related

Activity

  1. zackees commented on Jun 12, 2026

    @zackees
    MemberAuthor

    Status: both phases have landed.

    • Phase 1 (chore(ci): unify soldr versions, per-target cache keys, drop name-only ARM wrappers #171, merged): removed the broken version: 0.7.53 soldr pins, per-target cache-key-suffix, deleted the three name-only Windows ARM lint/unit/integration wrappers (~30 min/push saved outright).
    • Phase 2 (chore(ci): reuse CI artifacts in auto-release, cache dylint toolchain #172, merged): CI build wrappers now pass explicit rust-targets (release-identical artifacts; also fixed the macOS-x86-was-actually-arm64 bug), auto-release reuses CI artifacts for the release SHA with per-platform build fallback (~80 min saved per version bump), and _lint.yml caches the dylint nightly toolchain + binaries (10+ min fixed cost per lint job, now cache-hit) with the bare rustup call routed through soldr rustup.
    • Deliberately deferred: warm-then-fan-out restructure — with unified cache keys, cross-push cache sharing already covers most duplicate dep compilation, and a needs: warm gate would serialize the pipeline and worsen wall-clock latency.

    Remaining acceptance item: record the measured per-push compute drop vs the ~207-min baseline once a couple of post-merge pushes have populated the new caches.

  2. zackees commented on Jul 14, 2026

    @zackees
    MemberAuthor

    Validation on current main confirms issue #170 is complete.

    All checks for the measured post-phase push were successful. Closing this tracking issue; warm-then-fan-out remains deliberately deferred per the maintainer decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions