## Summary
- Provision Linux Miniconda explicitly instead of assuming `$CONDA` is
preinstalled, without creating or activating an extra environment.
Repair both PR workflows and baseline publishers.
- Replace retired macOS 14 runners with supported macOS 15 ARM64
runners, and run x86_64 workloads on `macos-15-intel`. Match Conda
installers to native architecture and retain both architectures'
functional/build coverage.
- Measure the exact PR base afresh. Coverage uses separate clean
base/head jobs because Unix Conda CI tests create global environments
and install Mamba into base. Performance builds each revision and
harness in its own worktree, then measures both on the same runner
against the same discovery workspace and fixtures.
- Validate actual source SHAs, run-attempt-qualified coverage artifacts,
matching nonempty measurement contexts, and successful
test/proof/extraction outcomes before invoking the unchanged
comparators.
- Address review feedback by extracting argument-driven
staging/capture/comparison into `scripts/quality_measurement.py` and
adding a distinct native ARM64 lane to both performance workflows while
retaining Intel.
## Root cause and impact
Before this repair, Linux jobs failed before testing because Ubuntu
26.04 no longer supplied the Conda installation assumed by the
workflows. The cancelled macOS jobs never received a runner: their macOS
14 configuration retired on November 2.
The original Windows numerical coverage failure compared Rust
1.99.1/LLVM 23.1.1 with Rust 1.100.0/LLVM 23.1.5 and different runner
images. With the same 96 Rust source files, LCOV denominators changed
from LF 23,480/FNF 2,231 to LF 23,288/FNF 2,254. This demonstrates
measurement-environment drift, not that the compiler alone explains
every changed hit.
The original macOS startup-P95 outlier is not treated as a proven
product defect. Its one unchanged-head rerun passed the timing budgets
but failed inventory matching: 12 environments versus the historical
baseline's 11. Fresh comparisons address that comparability problem
without discarding slow samples or normalizing inventories away.
## Preserved invariants and review follow-up
- No Rust/product, regression-budget, or snapshot-schema changes. The
0.01 percentage-point coverage tolerance, all performance budgets,
required samples, and inventory checks remain intact.
- Native ARM64 macOS coverage retains its default-feature workload and
same-runner exact-base comparison. Installed-manager fixtures,
environments without Python, subprocess proof, and fail-closed reporting
remain required.
- The extracted helper stages only the tested workflow's two immutable
script blobs plus a revision marker outside the discovery tree before
coverage checks out the measured source. An older base need not contain
the new helper. `helper_revision` and the measured source SHA are
validated separately; head Rust source/binaries are never substituted
for the base.
- Coverage and performance share common capture/provenance logic but
retain their different required tools and fixtures. Numerical gates
remain in the unchanged `quality_snapshot.py`.
- Native ARM64 performance has its own artifacts/comment lane and
freshly measured exact base. It uses the existing macOS budgets; Intel
remains included.
- Historical baseline workflows remain publishers for inspection. PR
gates no longer depend on their older artifacts. Fresh comparisons add a
real base measurement per platform.
- Separate from #578: its branch and installer-pinning changes are
untouched. No merge is requested.
## Current validation
Current head: `f10594fd644f112fa609dce548feab29d0dd4c92`, a new
**locally GPG-signed** commit preserving parent `31901b8`. Both local
GnuPG verification and GitHub's OpenPGP verification succeed. The
trusted-team-key signing policy recognizes the signature; `VS Code PR
Check` now passes with the required collaborator approval.
Actual measurements use exact base
`02dd29e49e3c71d4ec06e1072d6331494a12e01d` and tested merge
`d6d49a1ad7080a4e9bcda6259cd60a11fa3ab384` containing the current head.
- Local Python suite: 116 tests, one skipped. The parent independently
ran all 17 quality-workflow tests from the immutable signed revision.
Tests exercise the real CLI and reject wrong revisions, missing staged
helpers/dependencies, mismatched contexts, failed execution/extraction,
and accidental reuse of head as base.
- actionlint passes all three follow-up workflows; the original
seven-workflow repair was also linted.
- [Refactored Linux/Windows
coverage](https://github.com/microsoft/python-environment-tools/actions/runs/43964532818)
passes. Downloaded artifacts verify the old-base/new-helper case,
matching contexts, and required zero-idle/positive-info
handler/transport/writer witnesses for both revisions. Line deltas are
Linux +0.004445pp and Windows -0.004294pp; function deltas are zero,
within unchanged budgets.
- [All four paired performance
lanes](https://github.com/microsoft/python-environment-tools/actions/runs/43964532800)
pass all 12 gates with all nine distributions containing ten samples
each. Base/head environment inventories match: Linux 5/5, Windows 8/8,
Intel macOS 10/10, ARM64 macOS 12/12; managers are 1/1 throughout. ARM64
startup P95 is 73ms base versus 65ms head.
- [The updated performance
publisher](https://github.com/microsoft/python-environment-tools/actions/runs/43964849683)
passes on all four platforms. Its ARM64 VM independently has 11
environments/1 manager; that legitimate cross-run difference is
retained, not compared against the paired workload or normalized away.
## Remaining CI blockers
The review changes are pushed and all four reviewer threads have replies
and are resolved, but **the PR is not fully green**:
- [Native ARM64
coverage](https://github.com/microsoft/python-environment-tools/actions/runs/43964532819)
passes functional tests, isolated proof, and reporting but fails the raw
line gate by -0.031pp. Exactly one unchanged diagnostic rerun also
failed: head LH/LF 18,146/22,512 versus base 18,153/22,512, with
functions identical at 1,826/2,205. The seven-hit discrepancy moved
between unchanged source paths across attempts; the second attempt's
differing output.rs summaries have identical positive DA line sets. Both
failed artifacts are retained. This is unresolved runtime/summary-hit
variability, not a missing helper or removed test. No further rerun or
tolerance change was made.
- [Intel macOS functional
tests](https://github.com/microsoft/python-environment-tools/actions/runs/43964532730/job/133640357939)
fail `verify_validity_of_discovered_envs` at `ci_test.rs:629`: full
discovery includes one additional hosted-toolcache x64 Python symlink
that direct `try_from` does not return. Interpreter
identity/version/prefix/manager/architecture agree. Existing CI
normalization excludes ARM64 toolcache aliases but not x64. The initial
repair passed on the same image/Python version; why the extra alias
appears in this run remains unproven. This follow-up changes neither
`pr-check.yml` nor Rust and that job does not invoke the helper. No
product-locator workaround or blanket retry was made.
These remaining reliability/alias-causality issues are reported
explicitly rather than weakening gates or silently expanding the
maintenance-feedback fix.
---------
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pin the remaining mutable
taiki-e/install-action@cargo-llvm-covreference in all three coverage workflows (coverage.yml,coverage-baseline.yml, andcoverage-macos.yml) toab49d3c64680143e184118bed66a46419fc15519, the commit for v2.90.5. Add an explicittool: cargo-llvm-covinput and a human-readable version comment.Use the versioned release rather than the generated tool-tag commit: the action maintainer discourages pinning tool tags by hash because those generated commits become unreferenced after subsequent releases. The selected release commit is the parent of the current tool-tag commit, and their only difference is the tool input default; specifying that input explicitly preserves the existing installation behavior.
All 60 action references across the repository's nine workflows now use full-length commit SHAs. Existing weekly grouped Dependabot updates and the seven-day cooldown are unchanged.