Repository navigation
Link node_modules instead of copying it, and call jest directly (~4x faster) - #1038
Conversation
Each run copied the runner's node_modules (about 26,000 files, 148 MB) into the solution folder. That took about 1.2 s with a warm disk cache and about 15.6 s with a cold one, which matches the 18 s median run time seen in production. The runner now links the folder instead. The runner also called jest through corepack and pnpm, which started two extra Node processes, and listed node_modules/.bin and the solution folder on every run. It now calls node_modules/.bin/jest directly and skips the listings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@SleeplessByte I think you said on Discord you didn't think this was a good solution. Is there something you'd like us to investigate instead, or happy to push it forward? |
|
It didn't work, but that's how many nest versions ago |
|
I'm not sure what you mean. We've run this in production on ~200 solutions and its 4x faster. Which bit doesn't work? |
I am responding to your "I think you said on Discord". When we discussed it this solution did not work. In fact we tried it together for a run, when I asked you for the error logs. If it now works, I am happy with this solution 👍🏽 |
|
Thanks DJ. Yes, it works now. All 169 example solutions give identical results to main (status, messages and per-test output), and CI passes on Node 24 and 26. |
|
Thanks for merging, DJ. Since the deploy at 21:40 UTC, the median JavaScript test run in production has gone from about 17.5s to 2.7s, and the slowest 5% from about 24s to 4.8s. |
|
That makes no sense because the majority should run in the web browser, not use the test runner. Those definitely do not take 17.5s, let alone 3. |
|
Those numbers only cover runs that go through this server-side test runner, from the tooling job logs: about 1,250 JavaScript runs a day over the last four days. Runs in the browser aren't included. |
|
So how many runs are there through the browser, and then update the numbers? |
Hello 🙂 I'm working with Claude to try and get our tooling faster.
This PR makes this project around 4x faster. I've run it in production to benchmark things in the real world against the most recent few hundred solutions, so I'm confident that it works robustly. However, I've not written the code. I have tried my hardest to get Claude to write sensible messages and comments that explains stuff, but I'd ask for your patience in the fact I'm sending AI code to you. Everything below here is from your friendly neighbourhood LLM...
What changed
The runner now links
node_modulesinto the solution folder instead of copying it. Each run copied the runner'snode_modules(about 26,000 files, 148 MB) into the solution folder before running jest. On a production machine, that copy took about 1.2s when the files were already in the disk cache, and about 15.6s when they weren't. Production machines hold the Docker images for every Exercism language, so the JavaScript files are often not in the disk cache. This slow copy accounts for most of the 18s median run time in production. The link points at the runner's ownnode_modules, so jest, babel and the other packages load from the image the same way they did when the folder was copied.Two smaller changes save about one more second:
node_modules/.bin/jestdirectly. Calling jest throughcorepack pnpm jeststarted two extra Node processes (corepack, then pnpm) before jest itself. Thecorepack enable pnpmstep started a third.ls -alonnode_modules/.binand the solution folder on every run. These listings only helped with debugging.This has been tested in production, and has reduced the median run from 18.3s to 4.3s.
🤖 Generated with Claude Code