Skip to content

ci: gate main on the three regression suites, and make a skipped suite a failure - #418

Open
swapnilpaliwal-sd wants to merge 8 commits into
engine-prebuiltfrom
ci/github-actions
Open

swapnilpaliwal-sd wants to merge 8 commits into
engine-prebuiltfrom
ci/github-actions

Conversation

@swapnilpaliwal-sd

@swapnilpaliwal-sd swapnilpaliwal-sd commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #417

What changed

main gets a gate. Four required tiers, cheapest first: build and typecheck (parser and
driver, from this repository); the solver-free repo invariants (no-ignored-fixtures, the
staging guard across all four front ends, the engine's base declarations against the
parser's generated schema, client→library coverage floors, shell syntax); the java /
typescript / python / javascript regression suites in parallel; and every language's
engine built on every platform through the reusable build-engines.yml from #478, the
same artifacts publish-npm ships, built here without publishing.

Based on engine-prebuilt (#455, which now carries #478), so the build workflow it
calls exists on the base. Lands after the C# engine, together with that base.

The part that needed care

These suites are written for a developer machine, where a missing dependency is a good
reason to step aside: they print SKIP and exit 77. On CI that is the one outcome
that must never be tolerated, a gate that opens when its input is missing is worse than
no gate, because the green is read as evidence by whoever comes next.

Two instances, both of which would have gone unnoticed:

  • the parser is a separate repository; without it all three suites exit 77
  • test/java/torture reads class files with java.lang.classfile, which is not final
    before JDK 24. On an older JDK it exits 77 and the whole ten-family oracle does not
    run, while the suite still reports success

.github/scripts/run-suite.sh turns every shape of did not actually run into a
failure: exit 77, any sub-harness that swallowed its own 77, and a suite reporting
passed 0, which means the case loop matched nothing and the exit status says nothing.
CI supplies JDK 24 and pinned Python 3.10 rather than relaxing that check.

Ground truth, not only goldens

A golden says "the same as last time", which a wrong answer satisfies perfectly well as
long as it was wrong last time too. Java and TypeScript therefore run with --oracle,
scored against the toolchain that defines the language, javac and javap, and the
TypeScript compiler, with no third-party analyzer and no third-party library downloaded
to do it. A case whose ground truth would need an external classpath reports itself
unscored rather than pulling one in.

Python is goldens-only for now: its ground truth is frozen CPython output authored by a
separate harness that CI cannot reach yet. That is a weaker check than the other two
legs, and the workflow says so where it matters.

Pins, and why

  • the parser is in this repository (parser/) since the reshape, built by npm install's
    prepare script; there is no longer a separate checkout, token or commit pin, and the
    nightly drift workflow is gone with them. The goldens are still a function of both the
    rules and the IR, now both are in one diff.
  • Soufflé 2.5, verified against the checksum upstream published for that release.
    Soufflé is the solver the engine is compiled and linked against, so what CI links is
    decided in the workflow file rather than by whatever the archive serves that day.

Branch protection

.github/scripts/protect-main.sh holds the ruleset for main: pull request required,
an approving review, code-owner review, CI green against an up-to-date branch, linear
history, no force-push, no deletion, and no bypass actors. It is idempotent and applied
separately from this pull request.

main-guard.yml reports any commit that reaches main without a pull request. It is a
backstop, not the rule, a workflow runs after a push has been accepted, so it can record
a direct push but never refuse one.

Evidence

Run locally through .github/scripts/run-suite.sh, on this branch, with the in-repo parser:

  • java, --oracle --no-torture: passed 39 failed 0 (torture EXCLUDED, printed as such)
  • typescript, --oracle: passed 53, failed 0
  • python (goldens; the CPython harness is not reachable from CI): passed 15 failed 0
  • javascript, --oracle: passed: 19 failed: 0, tsc over allowJs/checkJs from this
    repo's devDependencies; the wrapper reads the JS suite's passed: line too
  • the wrapper was verified to refuse each skip shape the harnesses emit, and to refuse to
    start when parser/dist/index.js is absent
  • npm install, not npm ci: no lock file is committed (bundle stage runs from any working directory (tsx --tsconfig); lock files uncommitted #476)

This pull request's own CI run exercises the workflow, since pull_request evaluates it
from the merge ref.

Checklist

  • All four suites pass locally through the CI wrapper with the in-repo parser
  • No golden moved, this changes no rule
  • Labelled and assigned

Two faults found while getting the suites onto a clean machine

The java torture families cannot be scored without the platform IR. They call
java.util.List, Map and the functional interfaces, so the harness stages the JVM
platform IR as a library. Removing it takes the missing-edge census from 10 edges to 23
and recall to 0.847, the harness itself warns that the score then measures the staging,
not the rules. That IR is 1.8 GB and is built from a JDK source checkout, so it cannot
live in a repository, a cache, or a runner. CI passes --no-torture and the suite
prints EXCLUDED, because an excluded family must never be mistakeable for one that
passed. Java client->library resolution is still covered by the six cases shipping a
lib-src/ stub library.

Client->library coverage could vanish unnoticed. Delete one of those goldens and
nothing breaks: the case still runs, still passes, and quietly stops making the claim.
test/tools/lib-coverage.sh pins the shape, both goldens per TypeScript lib case, a
golden per java stub case, floors that cannot fall without being lowered in the same
commit. Parser-free, runs in seconds.

Coverage as it now stands, all of it running in CI:

front end client->client client->library
typescript 23 cases 23 cases, solved twice, the delta IS the mapping
java 39 cases 6 cases with a lib-src/ stub library
python 12 cases + 2 whole-project fixtures torture (client + lib)

CI status on this branch: green

job result time
build & typecheck pass 22s
repo invariants (5 checks) pass 7s
engine (java) passed 39 failed 0 3m05s
engine (typescript) passed 40, failed 0 3m47s
engine (python) passed 15 failed 0 3m26s
CI (the required gate) pass 2s

Under four minutes wall clock, and scoring against ground truth rather than only the
goldens, java reports per case oracle=15 engine=15 agree=15 missing=0, typescript the
same with the client->library half beside it, and the python torture links client and lib
for real at oracle=623 engine=613 agree=577. The java torture line reads
EXCLUDED (--no-torture), visible and never mistakable for a pass.

Two faults a clean machine found that a developer machine could not

The python torture family scored nothing on 3.10. CI reported only trace failed,
because the harness sends the tracer to /dev/null. Run directly the cause is exact:
client/f27_with_target.py imports typing.Self, which is 3.11+ (PEP 673), so the
tracer died at import and the only python coverage of a library boundary quietly
measured nothing. 3.10 cannot simply be dropped, the tier-1 attribution preflight reads
CPython opcodes, whose shapes are not stable across minor versions, so CI installs both
and prints both versions.

Every credential fault reports as remote: Write access to repository not granted,
whatever the real cause was. It is not about write access. CI now asks the API first and
names the fault, invalid, policy-blocked, or invisible, and reports how many of the
organisation's repositories the credential can actually reach, because a token owned by
a personal account looks identical to an unapproved one from the outside.

…e a failure

main had no gate. The suites only ran when somebody remembered to run them, and
running them is not cheap enough to be reliable as a habit.

Three required tiers: build and typecheck; the parser-free repo invariants
(no-ignored-fixtures, the staging guard for all three front ends, shell syntax);
and the java / python / typescript suites in parallel.

The part that needed care is that these suites are written for a laptop, where a
missing dependency is a good reason to step aside: they print SKIP and exit 77.
On CI that is the one outcome that must never be tolerated, because a gate that
opens when its input is missing is worse than no gate — the green gets read as
evidence. Two instances that would otherwise have gone unnoticed:

  - the parser is an external repository, and without it all three suites exit 77;
  - test/java/torture reads class files with java.lang.classfile, which is not
    final before JDK 24, so on the runner's default JDK the ten-family oracle
    exits 77 while the suite still reports success.

So .github/scripts/run-suite.sh converts every shape of "did not actually run"
into a failure: exit 77, any sub-harness that swallowed its own 77, and a suite
that reports "passed 0" and therefore asserted nothing. CI installs JDK 24 and
pinned Python 3.10 rather than relaxing that check.

The parser is pinned by SHA in .github/parser-ref. The goldens are a function of
both these rules and the IR, so tracking the parser's main branch would let a
change in the other repository turn this one red with no commit here to point at,
and would let a parser regression quietly become the new expectation. A nightly
workflow runs against the parser's main and opens an issue when the pin falls
behind, so the pin cannot rot unnoticed.

Soufflé is pinned to 2.5 for the same reason: its evaluation changes between
minor versions, and an unpinned solver makes every golden a moving target.

Branch protection is not applied here because it cannot be: rulesets are refused
with 403 on a private repository in a Free organisation. It is written as
.github/scripts/protect-main.sh, to run on the day this repository goes public,
when rulesets become free. Until then main-guard.yml records any commit that
reaches main without a pull request — detection rather than prevention, and the
file says so and says to delete it once the ruleset exists.

Also adds the contribution path for people who are not maintainers: issue
templates, a pull request template, CONTRIBUTING.md, CODEOWNERS, and build
badges on the README.

Fixes #417

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@swapnilpaliwal-sd swapnilpaliwal-sd added build Build, packaging and developer setup platform OS / toolchain portability test Test, fixture or gate coverage only labels Sep 12, 2026
@swapnilpaliwal-sd swapnilpaliwal-sd self-assigned this Sep 12, 2026
Follow-ups on the review of the previous commit.

The workflow was invalid and no job had ever started: a GitHub expression may
only delimit strings with single quotes, and join(needs.*.result, " ") used
double ones, which invalidates the whole file. actionlint catches this class of
fault in a second and would have caught it before the push.

Ground truth, not only goldens. A golden says "the same as last time", which a
wrong answer satisfies perfectly well as long as it was wrong last time too, so
java and typescript now run with --oracle: scored against javac and javap, and
against the TypeScript compiler. That needs `npm ci` in the engine job — the
compiler comes from this repo's own devDependencies, and without it every case
reports "oracle refused" while the run still looks like it did something.

No third-party library is downloaded to build ground truth. A case whose oracle
would need an external classpath reports itself unscored instead.

Soufflé is the solver the engine is compiled and linked against, so it is pinned
and its package verified against the checksum upstream published for that
release: what CI links is decided in the workflow file rather than by whatever
the archive serves that day. The package is cached, so a run does not depend on
that download succeeding twice.

The parser is fetched with a token rather than a deploy key.

CODEOWNERS now lists everyone on the project. CONTRIBUTING.md is removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@swapnilpaliwal-sd

Copy link
Copy Markdown
Contributor Author

This Fails. Working on this.

…ary coverage

Two faults found while getting the suites to run on a clean machine.

THE TORTURE FAMILIES CANNOT BE SCORED WITHOUT THE PLATFORM IR. The harness
stages the JVM platform IR as a library because half the families call
java.util.List, Map and the functional interfaces; with it absent those
receivers cannot be typed by any rule. Measured: the missing-edge census goes
from 10 to 23 and recall vs possible falls to 0.847. The harness already says
this in its own warning — the score then measures the staging, not the rules.

That IR is 1.8 GB and is built from a JDK source checkout, so it cannot live in
a repository, a cache, or a runner. Running the families anyway would paint a
red that reads as a regression in whatever change happened to meet it, so CI
passes --no-torture and the suite PRINTS that the families were excluded. An
excluded family must never be mistakeable for one that passed.

Java client->library resolution is still covered: six cases ship their own stub
library in lib-src/ and are solved with it as --library.

CLIENT->LIBRARY COVERAGE CAN VANISH WITHOUT ANYTHING NOTICING. Most assertions
here are client->client. The client->library half rests on fewer fixtures, and
deleting one of its goldens does not break anything: the case still runs, still
passes, and quietly stops making the claim. A suite can only check the
assertions it still has.

test/tools/lib-coverage.sh pins the shape of that coverage — every TypeScript
case with a lib/ has BOTH goldens, since the delta between them IS the
client->library mapping; every Java case with a lib-src/ has its golden; and the
counts cannot fall without lowering a floor in the same commit. It needs no
parser and no solver, so it runs in seconds alongside the other invariants.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
swapnilpaliwal-sd and others added 3 commits September 12, 2026 09:02
The secret says what it is rather than leaving "token" to carry the meaning.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
git reports every credential problem against a private repository as
"remote: Write access to repository not granted ... 403", whatever the real
cause was. It is not about write access, and the token is often not the thing
that is wrong, so the message sends you looking in the wrong place.

Ask the API first and name the fault: 401 invalid, 403 policy, 404 invisible —
and 404 is what GitHub returns rather than confirming a private repository
exists, so it is equally what a token with the wrong RESOURCE OWNER looks like.
That last one is the trap, so the step also reports how many repositories the
credential can reach and how many are the organisation's. None means the token
belongs to a personal account.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The torture family failed in CI with nothing but "trace failed", because the
harness sends the tracer's output to /dev/null. Run directly, the cause is
exact: client/f27_with_target.py does `from typing import Self`, and typing.Self
is 3.11+ (PEP 673), so on 3.10 the tracer dies at import and the whole
client+lib family — the only python coverage of a library boundary — scores
nothing.

3.10 cannot simply be dropped: the tier-1 attribution preflight reads CPython
opcodes, whose shapes are not stable across minor versions, and it resolves
python3.10 by name. So both are installed, 3.12 second because the later
setup-python wins for plain python3, and a step prints both so a future
divergence is visible in the log rather than inferred from a failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@swapnilpaliwal-sd

swapnilpaliwal-sd commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor Author

Build succeeds! This has access to the parser repo (temporarily). Remove it once these two repos are merged into one.

@swapnilpaliwal-sd

Copy link
Copy Markdown
Contributor Author

Keeping this open because we are adding more languages and more suites.

@swapnilpaliwal-sd

Copy link
Copy Markdown
Contributor Author

For the update to the current tree (this branch predates the reshape, PR #469):

  1. Paths: test/… is now graph/test/…; one leg per language via bin/axiomcode test <lang> also proves the dispatcher.
  2. JavaScript leg (bin/axiomcode test javascript) — the engine is on main since javascript: call-graph engine for the JavaScript front end (on reshape) #472.
  3. Parser suites (bin/axiomcode test parser) — the parser lives in parser/ now.
  4. A fresh-clone job: npm install && npm run build from a clean checkout. That exact failure reached main yesterday (build: exclude graph/test fixtures from the engine's compile, a fresh clone could not npm install #477) because nothing ran it.
  5. Node 22 or newer: the bundle stage needs node:sqlite; the branch pins 20.
  6. name: build if the README badge should read "build" (it shows the workflow's name).

…layout with the in-repo parser, four suites (javascript added), the decls-vs-schema drift check, and every platform's engine build through build-engines.yml

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…and uses no npm cache

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@swapnilpaliwal-sd

Copy link
Copy Markdown
Contributor Author

Three items from #579 (closed in favour of this) worth folding in:

  1. Java leg with --oracle --no-torture: the torture families need the JVM platform IR (1.8 GB, built from a JDK checkout), so on CI they fail by construction and read as a regression. ci: gate main, install, invariants, four engine suites, parser suites; skipped suites fail #579 added the --no-torture flag to graph/test/java/run-tests.sh (prints EXCLUDED); cherry-pick origin/ci-gate -- graph/test/java/run-tests.sh.
  2. Hygiene: graph/test/tools/bundle-test.sh (one schema across languages, SCHEMA.md current) and graph/test/tools/engine-id-test.sh (the id is path-independent and rule-sensitive). Both run in seconds without Soufflé.
  3. name: build on the workflow, so the README badge reads "build" rather than "CI".

@swapnilpaliwal-sd

Copy link
Copy Markdown
Contributor Author

On hold: not to be merged. The repository owner has held the npm engine packaging, the bundle change and the CI gating from main for now.

Recorded here because several sessions are working this repo concurrently and can merge. Please do not merge this, and do not merge it on someone else's behalf. It is not a review finding and says nothing about the change's quality.

swapnilpaliwal-sd added a commit that referenced this pull request Sep 24, 2026
#1258 dropped both when it restructured the header, leaving Build, License and
Node. They are status, not decoration: engines-not-yet-published is the one line
that tells a reader why the Requirements paragraph still asks for Soufflé, and
nightly-not-yet-enabled says the scheduled run is not there rather than passing.

Restored as they were, static shields rather than workflow badges — which is
deliberate, per the commit that made the nightly badge render by not asking for a
run that has not happened.

Worth a look before release: the engines badge points at #478, which is merged.
The machinery landed; the publish has not run. #418 is where the CI arrives, and
is the honest target for both.
swapnilpaliwal-sd added a commit that referenced this pull request Sep 25, 2026
Carries #418 onto current main as files rather than a rebase: its branch merged
engine-prebuilt, whose other changes main already has, so only .github/,
graph/test/tools/lib-coverage.sh and the java suite's --no-torture flag were new.

Beyond #418:
- C# joins the gate: a suite leg scored against the Roslyn oracle (.NET 8), its
  staging map and its decls-vs-parser-schema check.
- C# joins build-engines, so the engine packages carry all five languages.
- protect-main.sh points at the renamed repository and also makes v* tags
  immutable (creatable, never moved or deleted).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
swapnilpaliwal-sd added a commit that referenced this pull request Sep 25, 2026
Carries #418 onto current main as files rather than a rebase: its branch merged
engine-prebuilt, whose other changes main already has, so only .github/,
graph/test/tools/lib-coverage.sh and the java suite's --no-torture flag were new.

Beyond #418:
- C# joins the gate: a suite leg scored against the Roslyn oracle (.NET 8), its
  staging map and its decls-vs-parser-schema check.
- C# joins build-engines, so the engine packages carry all five languages.
- protect-main.sh points at the renamed repository and also makes v* tags
  immutable (creatable, never moved or deleted).
swapnilpaliwal-sd added a commit that referenced this pull request Sep 25, 2026
Carries #418 onto current main as files rather than a rebase: its branch merged
engine-prebuilt, whose other changes main already has, so only .github/,
graph/test/tools/lib-coverage.sh and the java suite's --no-torture flag were new.

Beyond #418:
- C# joins the gate: a suite leg scored against the Roslyn oracle (.NET 8), its
  staging map and its decls-vs-parser-schema check.
- C# joins build-engines, so the engine packages carry all five languages.
- protect-main.sh points at the renamed repository and also makes v* tags
  immutable (creatable, never moved or deleted).
swapnilpaliwal-sd added a commit that referenced this pull request Sep 25, 2026
…t release per bump; npm on publish (#1318)

* ci: gate main on build, repo invariants and all five language suites

Carries #418 onto current main as files rather than a rebase: its branch merged
engine-prebuilt, whose other changes main already has, so only .github/,
graph/test/tools/lib-coverage.sh and the java suite's --no-torture flag were new.

Beyond #418:
- C# joins the gate: a suite leg scored against the Roslyn oracle (.NET 8), its
  staging map and its decls-vs-parser-schema check.
- C# joins build-engines, so the engine packages carry all five languages.
- protect-main.sh points at the renamed repository and also makes v* tags
  immutable (creatable, never moved or deleted).

* release: one version everywhere, a tag and draft release per bump, npm on publish

- version.mjs sets and checks the version in all ten places that carry it
  (root, its engine pins, parser, gemini and the three plugin manifests).
- The version gate (from #1213) now also requires every manifest to agree, a
  bump to move forward, and the new version not to be tagged already. It only
  demands a bump once the base version is tagged, so work before the first
  release joins 0.1.0.
- release.yml: a push to main whose version has no tag gets one, plus a draft
  release with generated notes. Nothing is published.
- publish-npm.yml runs when that draft is published: checks tag == manifests,
  builds the engines, publishes them and then @axiomcode/code-graph (which was
  never published before), prereleases under `next`, skips versions already on
  the registry, and attaches the tarballs to the release.
- .github/RELEASING.md is the runbook.

* test/java: the oracle-agreement golden counts case 60

Case 60 (#1269) added a case both ground-truth oracles compare, without
re-recording the agreement golden, so `run-tests.sh --oracle` on main aborted
before running any case: 45 compared, 45 agreeing, against a golden of 44/44.
No disagreement changed.

* release: publish with the CLI_BINARY_PUBLISH secret, the npm token already set on the repo

* build: darwin-arm64 on the hosted macos-15 runner, every run

A public repository gets GitHub's standard runners free, Apple Silicon macOS
included, so the self-hosted runner and the macos switch that skipped it are
gone: CI and every publish build all four platforms.

* ci: no approval required; only the admin role merges into main

GitHub never lets an author approve their own pull request, so a required
approval would block the sole maintainer. The PR and CI rules still bind
everyone; a separate ruleset restricts updates to main to the admin role,
through a pull request only.

* ci: run on pushes to dev, which takes direct pushes

* release: merge main back into dev after every push to main

main squash-merges, so a promotion lands as a commit dev lacks and the next
dev->main PR would repeat it. A clean merge is pushed to dev; a conflict (a
hotfix that touched lines dev changed) pushes nothing and opens one issue with
the commands to resolve it by hand.

* Linux: an apostrophe in a # comment, and a sandbox PATH that kept /bin

Two faults only a Linux runner shows, found on the first CI run:

- Soufflé on Linux preprocesses with mcpp, which tokenises the text of a
  trailing # comment; "every on's listener" is an unterminated character
  constant there, so every TypeScript solve and the engine generate step
  failed. macOS's clang preprocessor accepts it. Reworded; all five
  languages' programs now pass mcpp.
- engine-id-test.sh hid souffle by dropping its directory from PATH, compared
  logically. On merged-/usr Ubuntu /bin -> /usr/bin survives that, and dropping
  /usr/bin itself would take bash with it. Directories are now compared with
  pwd -P and souffle's is replaced by a shadow holding everything else.

* test/tools: one souffle-hiding helper, correct on merged-/usr Linux

engine-package-test.sh carried the same PATH sandbox as engine-id-test.sh
and failed the same way on Ubuntu: /bin -> /usr/bin kept souffle visible, so
"no souffle" runs found it. Both now source hide-souffle.sh, which compares
directories with pwd -P and shadows souffle's directory instead of dropping
it. Proven on a simulated merged-/usr layout: the old block leaves souffle
visible, the helper hides it and keeps sh.

* ci: a docs-only change runs build and hygiene only; platform engines only when what they compile changed

The workflow still starts on every event, so the required CI check always
reports; a changes job classifies the diff and the engine suites and the
platform build skip themselves. The CI job accepts a skip only where changes
asked for one. Markdown under graph/ and parser/ still counts as code, since
graph/bundle/SCHEMA.md is generated and checked. Pushes to main, merge queues
and manual runs run everything.

* ci: dev is the default branch, a nightly from scratch, engines cached by ENGINE_ID

- dev takes every pull request: build, hygiene and the five suites. The
  four-platform engine build runs on the way into main (a promotion PR or a
  push to main) and in the nightly, not on every change to dev.
- protect-main.sh names refs/heads/main instead of ~DEFAULT_BRANCH, so main's
  protection stays on main now that dev is the default, and dev gets a
  ruleset that only forbids deleting it.
- nightly.yml: on nights dev changed, CI with fresh=true (no restored
  engines, every platform), then e2e-install.sh packs the tarballs, installs
  them into an empty project without Souffle and runs axiomcode in four
  languages, then npm publish --dry-run for every package. Publishes
  nothing; a failure opens an issue and the next green night closes it.
- The suite cache never hit: the driver writes to ~/.cache/axiomcode/souffle
  while CI saved .souffle-cache. The suites now point the driver there and
  key it by the language's ENGINE_ID.
- build-engines restores the previous engines per platform and recompiles
  only languages whose ENGINE_ID changed (about 3 min each at -O3); fresh
  (nightly, publish) restores nothing. Its Souffle download is now checked
  against the same SHA-512 as ci.yml.

* ci: cache keys cover the compile flags and, for -march=native, the CPU

ENGINE_ID hashes the rules and the Souffle version, not how the binary is
compiled, so a flag change reused binaries built with the old flags.
- build-engines: the hash of build-engines.yml (which holds the flags)
  prefixes both the key and the restore prefix, so a flag change restores
  nothing. Computed in generate, since the build jobs never check out.
- suites: the key adds the hash of run-souffle.sh (the driver's flags) and
  the runner's CPU model: the driver compiles with -march=native, and a
  binary built on one CPU can die with an illegal instruction on another.

* nightly: dev's daily status, published under the nightly dist-tag when green

- Runs every night, commits or not: the status is daily, and with no lock
  file a dependency release can break a fresh install with nothing committed.
- A green night publishes every package as <next>-nightly.<date>.g<sha>
  under `nightly`, never `latest`. The version is computed in the run and
  never committed or tagged; the g keeps an all-digit sha a valid semver
  identifier. Skipped when that commit is already the nightly, and until a
  first release exists, since npm makes a first publish `latest`.
- README: the static "engines: not yet published" and "nightly: not yet
  enabled" badges become the live npm version and nightly status.

* test/csharp: one compiled engine per run, not one per case; admins can merge

- devrun.sh caches its compiled engine beside the work dir it is given, and
  run-tests.sh hands every case a fresh one (rm -rf "$w"), so each of the 20
  cases and the cross-service cases recompiled the same engine, about 70s
  each: the suite took 32-38 min in CI. run-tests.sh now exports one
  AXIOM_CS_DEV_CACHE for the run (in CI, inside the saved engine cache). The
  cache is content-addressed by the rule text, so sharing it is safe.
  Locally: three cases, one compile and two reuses, 95s in total.
- main-merge-admins bypass is `always`: in `pull_request` mode GitHub refused
  the merge itself ("Cannot update this protected ref"), even for admins.
  protect-main still has no bypass, so PR and CI bind admins too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Build, packaging and developer setup platform OS / toolchain portability test Test, fixture or gate coverage only

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No CI gate on main: the regression suites never run before a merge, and a suite that skips itself reports success

1 participant