Repository navigation
Description
Context
fastled-wasm compiles FastLED sketches to WASM and shows them in a Tauri-hosted viewer pointed at a local HTTP server (see commit 0332caf and #154 for live build streaming). The user-facing CLI is the Rust binary in crates/fastled-cli. Python src/fastled/ is a thin shim.
The product owner wants lightweight, in-product testing hooks so an AI agent (or CI script) can verify what a compiled sketch actually renders — without bringing in Playwright, a headless Chromium, or any external browser-automation framework. The hooks must reuse the existing viewer process and HTTP server.
The implementing agent should be
fable-5withxhighreasoning effort.
Motivation — why not Playwright?
- Playwright requires a separate Node/Chromium toolchain that does not match the Tauri/WebView2/WKWebView runtime the user actually ships. Verifying a different browser is an unsound test.
- It bloats the install / CI footprint of a project whose value proposition is "one Rust binary, no Python required to run."
- The viewer process and HTTP server are already in
fastled; adding a few flags is far cheaper than an external harness. - An AI agent reasoning about a sketch needs a single deterministic command that produces a PNG and a log file, then exits with a clean status code. That maps directly to a flag, not a third-party tool.
Investigation Findings (what already exists)
The plumbing for almost every desired capability is in place:
| Capability | Status | Location |
|---|---|---|
| HTTP server with SSE build stream | shipped (#154) | crates/fastled-cli/src/server.rs |
Build-finish status event (status:success) |
shipped | compile_stream.rs::report_build_outcome |
| Viewer-side console/error/fetch capture | shipped behind FASTLED_VIEWER_LOGS=1 |
tauri_viewer.rs::LOG_FORWARD_SCRIPT |
POST /viewer-log endpoint (echoes to stderr) |
shipped | server.rs::viewer_log |
WebviewWindow::eval() for JS injection |
shipped, already used | tauri_viewer.rs |
| Viewer process liveness polling + clean shutdown | shipped | commands.rs::compile_and_serve |
| Screenshot API in Tauri 2.10 / wry 0.54 | NOT available | confirmed in registry sources |
| Headless mode for the viewer | not available; --internal-serve-dir-headless only skips the viewer |
Screenshot feasibility: wry 0.54 / tauri 2.10 expose no take_screenshot / capture_preview binding. Going OS-native (WebView2 ICoreWebView2_2::CapturePreview on Windows, WKWebView.takeSnapshot on macOS, GTK on Linux) is feasible but triples the surface area. The portable alternative is HTML5 canvas readback: the FastLED WASM renderer paints into a <canvas>, so canvas.toDataURL("image/png") produces a faithful frame. That PNG is POSTed back through a new /viewer-screenshot endpoint — same pattern as the existing /viewer-log. Recommend canvas readback. OS-native capture can be added later behind the same CLI surface without changing flags.
Proposed CLI API
Recommended: a single --test umbrella flag with structured sub-options
fastled examples/Blink \
--test \
--test-wait-secs=2.0 \
--test-screenshot=out/blink.png \
--test-log=out/blink.viewer.log \
--test-exit-on-error
Interval mode:
fastled examples/Blink \
--test \
--test-wait-secs=1.0 \
--test-interval-secs=0.5 \
--test-count=10 \
--test-screenshot=out/blink-{n:03}.png \
--test-log=out/blink.viewer.log
| Flag | Purpose |
|---|---|
--test |
Master switch. Implies no_interactive, disables file-watch rebuilds, enables canvas-readback + log-file sinks, and arms auto-exit. |
--test-wait-secs=<f> |
Seconds to wait after the "sketch ready" event (compile time excluded). Default 1.0. |
--test-screenshot=<PATH> |
Output PNG path. Supports {n}/{n:03} and {ts} placeholders for interval mode. Required if any screenshot is desired. |
--test-interval-secs=<f> |
Capture cadence. Triggers interval mode. |
--test-count=<N> |
Number of frames in interval mode. Mutually exclusive with --test-duration-secs. |
--test-duration-secs=<f> |
Total capture window in interval mode. |
--test-log=<PATH> |
Append viewer console/error log to file. Implies FASTLED_VIEWER_LOGS=1. |
--test-exit-on-error |
Nonzero exit if any console.error, page error, or failed fetch was observed during the test window. |
--test-timeout-secs=<f> |
Hard upper bound on total test duration (compile + wait + capture). Default 120. Exit code 124 on timeout. |
--test-ready-timeout-secs=<f> |
Time to wait for the "sketch ready" signal after status:success. Default 15. Exit code 125 on ready-timeout. |
--test-cmd=<file.js> |
Stretch goal. Inject a JS file after the ready event. |
Alternatives Considered
- Flat boolean:
--test-wait-then-take-picture-exit(user strawman). Self-documenting but does not extend to interval mode, separate log file, or exit-on-error semantics without a flag explosion. - Subcommand:
fastled test <sketch> [...]. Cleanest grouping, best--helpdiscoverability for agents, no collision with existing positionaldirectory. Strong runner-up. Rejected only because the current CLI is flag-driven (no subcommands today) and introducing one is a larger surgical change. If the implementing agent finds clap subcommand wiring straightforward, switching to this is preferred and explicitly endorsed. - JSON config:
--test-config=test.json. Keeps the flag surface tiny but is hostile to one-shot agent invocations (an agent now has to write a file first). - Reuse
--just-compile+ a separate--screenshot-after. Conflates compile-only mode with viewer launch; the viewer must be alive to capture frames. Rejected.
Why the recommended design wins
- Discoverable: every flag is prefixed
--test-*sofastled --help | grep testis exhaustive. - Scriptable for agents: a single command produces deterministic artifacts and a meaningful exit code; no orchestration of a long-running process is needed.
- Backward-compatible: zero collision with existing flags.
- Extensible: the stretch-goal
--test-cmdslots in without re-shaping anything.
Ready-signal Definition (when does the "wait X seconds" clock start?)
The "ready" event is the later of:
- SSE
build-streamevent withstatus:"success"(compile finished,index.htmlon disk), AND - Viewer has navigated to the real page (not the loading page) AND a
<canvas>exists in the DOM AND at least two requestAnimationFrame ticks have elapsed after canvas appearance.
Detection is JS-side, injected by the host after page load:
window.__fastled_test = window.__fastled_test || {};
const ready = new Promise(resolve => {
const tryFind = () => {
const c = document.querySelector('canvas');
if (c) {
requestAnimationFrame(() => requestAnimationFrame(() => resolve(c)));
} else {
setTimeout(tryFind, 50);
}
};
document.addEventListener('DOMContentLoaded', tryFind);
if (document.readyState !== 'loading') tryFind();
});
ready.then(c => {
window.__fastled_test.canvas = c;
fetch('/test-ready', { method: 'POST' });
});The host arms its --test-wait-secs timer when it receives POST /test-ready. If that never arrives within --test-ready-timeout-secs, the run exits with code 125. This explicitly excludes compile time from the wait, matching the stated requirement.
Console Log Capture
The existing LOG_FORWARD_SCRIPT is already correct: it intercepts console.{log,info,warn,error,debug}, window.onerror, unhandledrejection, and failed fetch. Adding --test-log=<PATH>:
- Sets
FASTLED_VIEWER_LOGS=1for the spawned viewer subprocess (already supported). - Opens the file in append mode in the host CLI before starting the viewer.
- Tees
viewer_logwrites to both stderr (current behavior) and the file. Each line is prefixed with a millisecond timestamp. - Records a marker line
[fastled-test] readywhen/test-readyfires and[fastled-test] screenshot=<path>after each capture, for easy correlation.
--test-exit-on-error scans the tee stream for any line whose console level is error, or that starts with window.onerror: / unhandledrejection: / fetch failed: / fetch error:, and sets the exit code accordingly.
Exit Semantics & Exit Codes
| Code | Meaning |
|---|---|
0 |
Test ran to completion. No errors observed (or --test-exit-on-error not set). |
1 |
Generic failure (compile failed, viewer failed to launch, IO error writing PNG/log). |
2 |
One or more page-side errors observed with --test-exit-on-error. |
124 |
Hit --test-timeout-secs. |
125 |
Did not observe ready signal within --test-ready-timeout-secs. |
Auto-exit semantics:
- After all screenshots are written, host calls
window.close()viaevaland then dropsViewerProcess(existing kill-on-job-close on Windows,ContainedProcessGroupon Unix). No lingering window or HTTP server. - Ctrl+C remains supported and exits with
130.
Screenshot Capture (implementation)
JS injected after /test-ready:
window.__fastled_test.capture = async function(name) {
const c = window.__fastled_test.canvas;
// toDataURL preserves the most recently composited frame for 2D/WebGL with
// preserveDrawingBuffer:true; for WebGL contexts created without it we
// request a synchronous readback via gl.readPixels in the same RAF tick.
const dataUrl = c.toDataURL('image/png');
await fetch('/viewer-screenshot?name=' + encodeURIComponent(name), {
method: 'POST',
headers: {'Content-Type': 'application/octet-stream'},
body: await (await fetch(dataUrl)).blob()
});
};New endpoint POST /viewer-screenshot?name=<basename> in server.rs decodes the body and writes it to the configured output directory. The host's capture scheduler is a simple tokio::time::interval driven by --test-interval-secs; it issues a WebviewWindow::eval("window.__fastled_test.capture('frame-N.png')") per tick.
WebGL caveat (CRITICAL): FastLED's WASM renderer may create its WebGL context without
preserveDrawingBuffer: true. In that casetoDataURLreturns a blank frame. Mitigation: the injected JS detectsgetContext("webgl"|"webgl2")and patches the constructor to forcepreserveDrawingBuffer: trueBEFORE the sketch's JS runs. Implementing agent must verify this with at least one WebGL-based example (e.g.,examples/NoisePlusPalette).
Stretch Goal — --test-cmd <file.js>
When --test-cmd is set, the host reads the JS file, base64-encodes it, and after /test-ready calls eval("(new Function(atob('...')))()"). A minimal host bridge is exposed to the injected script:
window.__fastled_test.screenshot(name); // forces a screenshot
window.__fastled_test.log(msg); // routed through /viewer-log
window.__fastled_test.done(exit_code); // POSTs /test-done, host exits
window.__fastled_test.fail(message); // shorthand for done(2) + logPOST /test-done is the IPC channel back to the host that completes the run. Without it, an injected script has no way to terminate the host early — the host would always run the full --test-wait-secs then exit.
Why deferred:
- Requires designing a stable JS-side API surface (versioned via
window.__fastled_test.version). - Introduces user-controlled JS evaluation against the FastLED page — needs an explicit security note ("only run scripts you trust; injected JS has full DOM access").
- The non-stretch features above already deliver the verification capability for >=80% of agent workflows (compile -> render -> screenshot -> check log).
Implementation Sketch
Files to touch (all in crates/fastled-cli):
src/cli.rs— add the--test*flags. Keep them under a#[command(next_help_heading="Production testing")]group for discoverability.src/lib.rs— early-branch intocommands::compile_and_testwhen--testis set.src/commands.rs— newcompile_and_test()derived fromcompile_and_serve(), with: no file watcher, deterministic shutdown, capture-scheduler task,--test-logtee. Do not duplicate the compile path — refactor the watch loop into aRunModeenum.src/server.rs— addPOST /viewer-ready,POST /viewer-screenshot,POST /test-donehandlers; pipe their events out via a newmpsc::Sender<TestEvent>field onAppState.src/tauri_viewer.rs— extendViewerOptionswithinject_test_runtime: bool; concatenate a newTEST_RUNTIME_SCRIPT(ready-detect + capture API + WebGLpreserveDrawingBufferpatch) onto the existingLOG_FORWARD_SCRIPT.src/main.rs— propagate test-runtime injection through--internal-viewer(new sibling flag--viewer-inject-test-runtime).
Feasibility notes from investigation:
WebviewWindow::evalis available in tauri 2.10 and is the only JS injection primitive needed — no IPC plugin required because all paths use HTTP fetch back to the same server. The architecture is symmetric with the existing log-forwarding feature, which is a strong sign it will land cleanly.- Tauri/wry have no screenshot binding in the versions in use; canvas readback is the correct choice.
- Auto-exit already works via
ViewerProcess::is_alivepolling; we just add a "test-completed" condition alongside the existing "viewer closed" condition.
Acceptance Criteria
-
fastled <sketch> --test --test-wait-secs=2 --test-screenshot=out.pngcompiles the sketch, waits 2 s after the sketch renders, writes a non-blank PNG, and exits0. Compile time is not counted toward the 2 s. -
--test-interval-secs=0.5 --test-count=10 --test-screenshot=out-{n:03}.pngwrites exactly 10 PNGs namedout-000.png...out-009.png. -
--test-log=run.logproduces a file containing at minimum the[fastled-test] readymarker and everyconsole.*line emitted by the sketch. - A sketch that calls
console.error("boom")with--test-exit-on-errorset exits with code2. - A sketch that fails to compile exits with code
1. A sketch that compiles but never renders a canvas within--test-ready-timeout-secsexits with code125. - No viewer window, viewer process, or HTTP server remains alive after the CLI exits (verified by job-object / process-group teardown).
- At least one WebGL-based example produces a non-blank screenshot (validates the
preserveDrawingBufferpatch). -
bash lintandsoldr cargo test --workspacepass. New unit tests cover: filename templating, exit-code mapping, ready-timeout, error-line classification. - Documentation updated in
README.md(one short "Testing your sketch from CI / an agent" section) andCLAUDE.mdConventions note pointing at the test flags. -
--test-cmdis not required for this PR; it lands behind a follow-up issue referencing this one.
Decisions
- Single
--testumbrella with prefixed sub-flags, not a subcommand. Smaller refactor, easier to extend. - Canvas
toDataURLover OS-native window capture. Portable, no per-platform code, faithful to what the user actually sees in the canvas. - JS injection over a Tauri IPC plugin. Reuses the existing
LOG_FORWARD_SCRIPTpattern; no new plugin/feature flag ontauri. - HTTP POST back to the same server, not Tauri IPC. The server is already there, the viewer already POSTs to it, and this keeps the design uniform.
- Patch
getContextto forcepreserveDrawingBuffer: truerather than swap WebGL for software rendering. Cheaper, doesn't change what the sketch sees. - Defer
--test-cmdto a follow-up; the bridge design is sketched here so the follow-up doesn't re-litigate it. - Exit-on-error is opt-in. A sketch that logs a benign warning shouldn't fail a screenshot run by default.
Activity
feat(cli): add production test hooks (#201)