Skip to content

Link node_modules instead of copying it, and call jest directly (~4x faster) - #1038

Merged
SleeplessByte merged 1 commit into
mainfrom
faster-test-runs
Nov 19, 2026
Merged

SleeplessByte merged 1 commit into
mainfrom
faster-test-runs

Conversation

@iHiD

@iHiD iHiD commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Hello 🙂 I'm working with Claude to try and get our tooling faster.

This PR makes this project around 4x faster. I've run it in production to benchmark things in the real world against the most recent few hundred solutions, so I'm confident that it works robustly. However, I've not written the code. I have tried my hardest to get Claude to write sensible messages and comments that explains stuff, but I'd ask for your patience in the fact I'm sending AI code to you. Everything below here is from your friendly neighbourhood LLM...


What changed

The runner now links node_modules into the solution folder instead of copying it. Each run copied the runner's node_modules (about 26,000 files, 148 MB) into the solution folder before running jest. On a production machine, that copy took about 1.2s when the files were already in the disk cache, and about 15.6s when they weren't. Production machines hold the Docker images for every Exercism language, so the JavaScript files are often not in the disk cache. This slow copy accounts for most of the 18s median run time in production. The link points at the runner's own node_modules, so jest, babel and the other packages load from the image the same way they did when the folder was copied.

Two smaller changes save about one more second:

  • The runner calls node_modules/.bin/jest directly. Calling jest through corepack pnpm jest started two extra Node processes (corepack, then pnpm) before jest itself. The corepack enable pnpm step started a third.
  • The runner no longer runs ls -al on node_modules/.bin and the solution folder on every run. These listings only helped with debugging.

This has been tested in production, and has reduced the median run from 18.3s to 4.3s.

🤖 Generated with Claude Code

Each run copied the runner's node_modules (about 26,000 files, 148 MB) into
the solution folder. That took about 1.2 s with a warm disk cache and about
15.6 s with a cold one, which matches the 18 s median run time seen in
production. The runner now links the folder instead.

The runner also called jest through corepack and pnpm, which started two
extra Node processes, and listed node_modules/.bin and the solution folder
on every run. It now calls node_modules/.bin/jest directly and skips the
listings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@iHiD
iHiD requested a review from a team as a code owner November 17, 2026 16:50
@iHiD iHiD changed the title Link node_modules instead of copying it, and call jest directly Link node_modules instead of copying it, and call jest directly (~3x faster) Nov 18, 2026
@iHiD iHiD changed the title Link node_modules instead of copying it, and call jest directly (~3x faster) Link node_modules instead of copying it, and call jest directly (~4x faster) Nov 18, 2026
@iHiD

iHiD commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

@SleeplessByte I think you said on Discord you didn't think this was a good solution. Is there something you'd like us to investigate instead, or happy to push it forward?

@SleeplessByte

Copy link
Copy Markdown
Member

It didn't work, but that's how many nest versions ago

@iHiD

iHiD commented Nov 19, 2026

Copy link
Copy Markdown
Member Author

I'm not sure what you mean. We've run this in production on ~200 solutions and its 4x faster. Which bit doesn't work?

@SleeplessByte

Copy link
Copy Markdown
Member

I'm not sure what you mean. We've run this in production on ~200 solutions and its 4x faster. Which bit doesn't work?

I am responding to your "I think you said on Discord". When we discussed it this solution did not work. In fact we tried it together for a run, when I asked you for the error logs. If it now works, I am happy with this solution 👍🏽

@ihid-assistant

Copy link
Copy Markdown
Contributor

Thanks DJ. Yes, it works now. All 169 example solutions give identical results to main (status, messages and per-test output), and CI passes on Node 24 and 26.

@SleeplessByte
SleeplessByte merged commit 670eb13 into main Nov 19, 2026
9 checks passed
@SleeplessByte
SleeplessByte deleted the faster-test-runs branch November 19, 2026 21:32
@ihid-assistant

Copy link
Copy Markdown
Contributor

Thanks for merging, DJ. Since the deploy at 21:40 UTC, the median JavaScript test run in production has gone from about 17.5s to 2.7s, and the slowest 5% from about 24s to 4.8s.

@SleeplessByte

Copy link
Copy Markdown
Member

That makes no sense because the majority should run in the web browser, not use the test runner. Those definitely do not take 17.5s, let alone 3.

@ihid-assistant

Copy link
Copy Markdown
Contributor

Those numbers only cover runs that go through this server-side test runner, from the tooling job logs: about 1,250 JavaScript runs a day over the last four days. Runs in the browser aren't included.

@SleeplessByte

Copy link
Copy Markdown
Member

So how many runs are there through the browser, and then update the numbers?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants