Skip to content

fix(ci): update scribe-template test to read from scribe-charter.md - #1154

Merged
tamirdresher merged 7 commits into
github:devfrom
obit91:obit91/1151-fix-scribe-template-test
May 25, 2026
Merged

tamirdresher merged 7 commits into
github:devfrom
obit91:obit91/1151-fix-scribe-template-test

Conversation

@obit91

@obit91 obit91 commented May 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes two CI test failures introduced by PR #1035 (coordinator slimming / context overflow sentinel) that were not caught before merge.

Closes #1151

Root Cause

PR #1035 extracted Scribe content and spawn-template content out of squad.agent.md into standalone reference files (scribe-charter.md, spawn-reference.md, after-agent-reference.md) but did not update the CI tests.

Changes

test/ci/scribe-template.test.ts

Was reading squad.agent.md with anchors that no longer exist after the Scribe section was extracted to scribe-charter.md. Now reads the correct file with:

  • Number- and format-agnostic end-marker in extractScribeTaskBlock — no longer brittle to step numbering or bold formatting changes
  • Numbered-line assertions — verify phrases appear on actual \d+. list items, not anywhere in the block
  • Corrected file header comment — previous comment incorrectly stated HEALTH REPORT and size thresholds were absent; they are present and now tested
  • New tests added: HEALTH REPORT emission documented, Tier 1 archival threshold (20KB), Tier 2 archival threshold (50KB) — 10 tests total (was 7)

test/ci/datetime-template.test.ts

Was counting CURRENT_DATETIME: lines in squad.agent.md only, expecting ≥4. After slimming, only 2 remain in squad.agent.md; the rest moved to spawn-reference.md and after-agent-reference.md. Fixed to combine all coordinator-owned template files for spawn-template assertions.

Also includes

A retroactive changeset for PR #1035 (merged without one), covering both @bradygaster/squad-cli and @bradygaster/squad-sdk at patch level.

Testing

All 10 tests in scribe-template.test.ts and all 5 tests in datetime-template.test.ts pass locally.

Note: Policy gate failures on this PR are due to the prerelease version (0.9.6-build.4) — that is addressed separately in PR #1155.

PR github#1035 moved the Scribe section from squad.agent.md into a standalone
scribe-charter.md. The CI test still pointed at squad.agent.md and used
anchors ('Tasks (in order):', 'Never speak to user.') and labels
(PRE-CHECK, HEALTH REPORT, GIT COMMIT, DECISIONS ARCHIVE, 20480) that no
longer exist there.

This commit:
- Changes readTemplate() to read .squad-templates/scribe-charter.md
- Updates extractScribeTaskBlock() to use anchors present in the charter:
  'After every substantial work session:' .. '6. **Never speak to the user.**'
- Rewrites the describe block and test assertions to match the charter's
  actual numbered-step structure (steps 1-6: log, merge inbox, deduplicate,
  propagate, commit, never speak)
- Preserves the HARD GATE enforcement invariant (still checked in content)
- Preserves commit-precedes-silence ordering invariant

Closes github#1151

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@obit91
obit91 marked this pull request as ready for review May 22, 2026 15:59
Copilot AI review requested due to automatic review settings May 22, 2026 15:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates CI tests to validate the new standalone Scribe charter (migrated from the previous squad agent template) and its expected task-step structure.

Changes:

  • Point tests at .squad-templates/scribe-charter.md instead of .squad-templates/squad.agent.md
  • Update extraction logic to pull the numbered “work session” task block from the charter
  • Replace prior PRE-CHECK / HEALTH REPORT / 20480-threshold assertions with charter-specific step assertions (including “Never speak to the user.” final step)

Comment thread test/ci/scribe-template.test.ts Resolved
Comment thread test/ci/scribe-template.test.ts Resolved
Comment thread test/ci/scribe-template.test.ts Resolved
Comment thread test/ci/scribe-template.test.ts Resolved
Comment thread test/ci/scribe-template.test.ts Resolved
obit91 and others added 3 commits May 22, 2026 19:12
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… sentinel)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PR github#1035 moved spawn templates out of squad.agent.md into on-demand
reference files (spawn-reference.md, after-agent-reference.md).
The datetime test was checking squad.agent.md only for CURRENT_DATETIME
wiring, which now fails because the count dropped from 4+ to 2.

Fix: combine squad.agent.md + spawn-reference.md + after-agent-reference.md
for the spawn-template assertions. The literal-substitution assertion also
now reads from spawn-reference.md where the example lives post-slimming.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix end-marker brittleness: extractScribeTaskBlock now locates the
  'Never speak to the user.' line by text search (number- and
  formatting-agnostic) instead of hard-coding '6. **...**'
- Fix numbered-step assertions: decision-inbox-merge, deduplication,
  and commit tests now verify the phrase appears on a line starting with
  '\d+.' rather than a plain substring search anywhere in the block
- Fix file-header comment: replace 'How I Work section' reference with
  the actual anchor marker 'After every substantial work session:'

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@obit91

obit91 commented May 22, 2026 •

Copy link
Copy Markdown
Contributor Author

@bradygaster @tamirdresher -- heads up on this PR context:

This fixes two CI test failures introduced by PR #1035 but not caught before merge:

  1. test/ci/scribe-template.test.ts -- was reading from squad.agent.md with anchors that no longer exist after Scribe was extracted to scribe-charter.md. Rewritten to target the correct file/structure.
  2. test/ci/datetime-template.test.ts -- was counting CURRENT_DATETIME: lines in squad.agent.md only, expecting >= 4. After slimming only 2 remain there; the rest moved to spawn-reference.md and after-agent-reference.md. Updated to combine all coordinator-owned template files for those assertions.

All 12 affected tests pass locally. Also includes a retroactive changeset for PR #1035 (merged without one) and a changeset for this fix.

Policy gate failures (0.9.6-build.4 prerelease version) are addressed in PR #1155 -- that PR needs a skip-version-check label from a maintainer.

obit91 and others added 2 commits May 22, 2026 19:44
…ld tests

The comment incorrectly stated that HEALTH REPORT and the archival size
thresholds were not present in the new scribe-charter.md. Both are present
(HEALTH REPORT as prose, thresholds as 20KB/50KB). Only PRE-CHECK and
GIT COMMIT as bold section labels are absent.

- Corrects the file header comment to accurately describe what changed
- Adds test: HEALTH REPORT emission is documented
- Adds test: Tier 1 archival threshold (20KB) is documented
- Adds test: Tier 2 archival threshold (50KB) is documented

10 tests now pass (was 7).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…thresholds)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tamirdresher
tamirdresher merged commit 490adf3 into github:dev May 25, 2026
5 of 6 checks passed
bradygaster pushed a commit that referenced this pull request May 26, 2026
…spawn layout

PR #1154 fixed the original CI failures from #1151 but two assertions
drifted again as the charter and spawn templates continued evolving:

- scribe-template.test.ts expected a numbered `Commit` step in the
  Scribe task block. The runtime-state-backend migration removed git
  commits from Scribe (mutable state is persisted via `squad_state_*`)
  and the charter now explicitly forbids Scribe from committing or
  switching branches. Replaced the commit-step assertions with a
  persistence-verification assertion (matches the `Verify persistence`n  through the runtime backend` step) plus a guard that the charter
  continues to forbid committing mutable squad state.

- datetime-template.test.ts expected the literal-substitution strings
  (`<literal CURRENT_DATETIME value from your prompt>` and `Substitute
  the actual CURRENT_DATETIME value`) to live in spawn-reference.md.
  They are still defined in the Full Spawn Template inline block in
  squad.agent.md. Widened the assertion to check across all
  coordinator-owned template files (squad.agent.md +
  spawn-reference.md + after-agent-reference.md) so it survives future
  relocations within that set.

All 16 tests across both files pass locally.

Related: #1151, #1154

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(ci): multiple CI tests fail after PR #1035 coordinator slimming

3 participants