Repository navigation
Require an AI cross-review before merge - #229
lwwmanning wants to merge 19 commits into
Conversation
Every pull request needs a Codex review of its head commit. A pull request whose footer or author names Codex also needs a Claude review. The gate sets the commit status "AI cross-review" for the main ruleset to require. Signed-off-by: Will Manning <will@willmanning.io>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
To use Codex here, create an environment for this repo. |
…view-f2ab62 Signed-off-by: Will Manning <will@willmanning.io> # Conflicts: # CLAUDE.md
CodeQL flagged the explicit checkout of the head commit as an untrusted checkout, because the workflow also runs on issue_comment. Signed-off-by: Will Manning <will@willmanning.io>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
CodeQL flags any job that runs the pull request's code in a workflow that also runs on issue_comment. ai-review.yml now runs on pull_request only, and ai-review-recheck.yml reruns main's copy of the gate on comments and reviews. The gate's token also gets pull-requests: write, which a comment on a pull request needs. Signed-off-by: Will Manning <will@willmanning.io>
|
@codex review |
🔦 Lighthouse audit
Run |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e51dd80774
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex's review found that the gate ran in the pull request's context: forks and Dependabot PRs got a read-only token, a PR could change the script that judged it, and fork PRs could never get a Claude review. The workflow now runs on pull_request_target, comments and reviews, and every job checks out main. Claude reads the PR through the API. The gate also asks Codex again after 20 minutes without a start (three times at most), retries a Claude review that ended without a verdict, and ignores dismissed Codex reviews. ci.yml runs the gate's tests. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 25fbcceb11
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The gate sets itself to pending before it reads the reviews, so a failed run cannot leave an old success in place. It fails when another open PR has the same head commit, because a commit status counts for every PR on that commit. A submitted Codex review of the head now counts even while the summary comment still says the review is running. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 20e6373921
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The Claude status name now holds the PR number, so a review of one PR does not count for another PR on the same commit. The gate counts Claude's blocking findings from its unresolved P0 and P1 threads, as it does for Codex, so resolving a thread and commenting clears it without a new push. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c5093392b2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A PR that moves to another base branch keeps its head commit, but its diff can change, so the gate posts a marker and fails until a new commit arrives. Only P0 and P1 threads count as evidence for Claude's counted findings. The gate reads every page of the commit's statuses, and counts only marker comments from github-actions[bot]. Signed-off-by: Will Manning <will@willmanning.io>
…view-f2ab62 Signed-off-by: Will Manning <will@willmanning.io> # Conflicts: # CLAUDE.md
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3516cffac1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Drop the pull_request_review trigger, which runs the PR's copy of the workflow. Claude reviews only PRs by authors with write access; an outside PR that Codex helped write needs a maintainer's approval of the head commit. Record a base change as a commit status before the notice comment. Mark a Claude review as started only after setup, and retry a review that stays pending for 45 minutes. Check at verdict time that each blocker Claude counts has an open thread. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d46d93d67d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
author_association calls a read-only organization member a MEMBER, so the gate now asks the permission API whether the author, or an approver of the head commit, can push. The Claude action can check out the PR's branch, so the job checks out main again before it records the verdict, and Claude reads CLAUDE.md from main. The gate's rerun after a review is now a job in the gate's concurrency group. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 576ce296d9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…odex's finding flag The first gate run on a head commit records the base branch as a status, and later runs compare against it, so a cancelled waiting run cannot lose a base change. An approval counts only while it is the reviewer's latest approval or request for changes. When Codex reports findings on the head commit but no Codex thread on that commit shows one, the gate fails. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 92517a3682
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A commit status belongs to the commit, so another PR with the same head would inherit it. The gate now runs on closed and sets its status back to pending unless the PR merged. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d313911250
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The gate job sets its status to pending before checkout and setup, so a failed setup cannot leave an earlier pass. A closed PR, merged or not, withdraws its status. The Claude job records a verdict only when main was checked out again. The base record holds the branch name's SHA-256, so a long name cannot be cut to match another. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 37b41373b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A review, or a PR closing beside a sibling with the same head, starts no gate run. Every 15 minutes a scheduled run from main evaluates each open, ready PR again under the gate's lock, and writes the status only when the result changed. A Codex review still in progress after an hour counts as stopped, so the gate may ask for it again. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cc31d561a4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…reviews The refresh starts the workflow by workflow_dispatch for a PR whose Claude review is due, because only a gate run can start the Claude job. The gate starts at most three Claude reviews per commit. A dispatched run has no PR context for the inline comment tool, so Claude can post findings through the review comments API. A dismissed Codex review of the head commit no longer counts through Codex's summary comment. Signed-off-by: Will Manning <will@willmanning.io>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f0e8231fc4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The Claude action refuses an actor without write access, and each refused start used one of the commit's three attempts. The gate now starts a review only for Codex, the workflow's own token, or an actor with write access; the scheduled refresh starts it for anyone else's event. Signed-off-by: Will Manning <will@willmanning.io>
A writer can post the commit status from a pull_request workflow, and the gate cannot count Codex's findings, only see whether it found any. Signed-off-by: Will Manning <will@willmanning.io>
…view-f2ab62 Signed-off-by: Will Manning <will@willmanning.io> # Conflicts: # CLAUDE.md
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 380114f349
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const baseChanged = | ||
| record !== undefined && record.description !== baseRecord(pr.base.ref); |
There was a problem hiding this comment.
Record the base commit as well as the branch
When the target branch receives new commits while the PR head is unchanged, pr.base.ref remains the same even though the merge base—and therefore the gh pr diff reviewed by Codex and Claude—can change. Subsequent gate runs leave baseChanged false and reuse reviews of the previous diff, allowing an unreviewed diff to pass. The fresh evidence beyond the earlier retargeting fix is that the persisted identity still contains only base.ref; record and compare the base SHA as well.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Not changed, and left open for a maintainer to decide. The reviewed diff is the head against its merge base with the base branch. While the head stays the same, new commits on main do not change that merge base, because the old one is still the nearest shared ancestor and the ruleset blocks history rewrites on main. There is one exception: main can take in commits that the head already has. Then the merge base moves forward, and the diff only loses changes that main already contains. Recording base.sha would void every open PR's reviews each time main moves, while the diff stays the same. A merge of main into the PR is a new head commit, and that already needs new reviews.
🤖 Addressed by Claude Code
What changes
This adds a merge gate for AI review to both sites' pull requests, in a new workflow,
ai-review.yml. The other checks stay as they are.Before, Codex reviewed some pull requests on its own, and Claude reviewed only when someone tagged
@claude. Neither review blocked a merge. After, the gate sets the commit statusAI cross-reviewon the head commit. Every pull request needs a clean Codex review of that commit. If the footer or the author names Codex, the pull request also needs a clean Claude review, which the workflow runs.A P0 or P1 finding fails the gate, so auto-fix can act on it. When Codex has not reviewed the head commit, the gate posts
@codex review. Renovate and Dependabot pull requests pass without review.After merge, the
mainruleset must requireAI cross-reviewbefore the gate blocks anything.How it was checked
bun test scripts/ai-review-gate.test.ts, the typecheck andbun run check:cipass. Dry runs against #216, #220, #225, #206 and #227 gave the expected status for each.git commit -s), for the DCO check.bun run check:cipasses.REUSE.toml.🤖 Generated with Claude Code