Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Agent Instructions

This file is read by Claude Code, Codex, and other coding agents that work on this repo. Codex agents should also see `CODEX.md` — that file is a single-line pointer back here so both agents read the same playbook.

## Layout

- Python package: `src/fastled/`
- Rust workspace: `crates/fastled-cli`, `crates/fastled-py`, `crates/fastled-tauri`
- Tests: `tests/unit/` (Python), `cargo test --workspace` (Rust)
- CI workflows: `.github/workflows/_*.yml`
- Dev scripts: `./install`, `./lint`, `./test`, `./build-wheel`, `./clean`

## Build cache — use soldr + zccache

This project uses **soldr** (build orchestrator) + **zccache** (compile cache). They are NOT optional for local Rust builds: a clean `cargo build -p fastled-py` recompiles PyO3 + reqwest + tokio + axum + clap from scratch and takes 8–10 minutes. With soldr/zccache the same build hits the cache and finishes in seconds.

- Binaries live at `/c/tools/python13/Scripts/{soldr,zccache}` on Windows, or wherever `uv pip install soldr` placed them.
- `soldr` wraps `cargo` and automatically uses `zccache` as `RUSTC_WRAPPER`.
- CI installs soldr via `zackees/setup-soldr@v0` (see workflows). Local dev should match.

**When you invoke cargo from an agent, prefer `soldr cargo …` over plain `cargo …`.** The `./_cargo` trampoline currently exec's plain `cargo` — that's a known gap (#76) and will be replaced. Until then, run cargo directly through soldr or you'll burn 10 minutes per build.

If `soldr` is not on PATH, fall back to plain `cargo` but warn the user that builds will be slow.

## Tests are slow — pick a narrow set

`bash test` runs the full pytest + cargo test suite. Some unit tests (`tests/unit/test_cli.py`, `tests/unit/test_build_service.py`) perform real WASM compiles via emscripten and can take 10+ minutes each.

For iterative work, run targeted tests:

```bash
uv run pytest tests/unit/test_string_diff.py tests/unit/test_select_sketch_directory.py -v
```

Skip `test_cli.py` unless you actually need to validate end-to-end WASM output:

```bash
uv run pytest tests/unit --ignore=tests/unit/test_cli.py -v
```

**Do not pipe pytest through `| tail -N`** if you want streaming output — it buffers until pytest exits, and a multi-minute pytest looks indistinguishable from a hang. Redirect to a file or use `-v`.

## Architecture notes

- The user-facing CLI is the Rust binary `fastled` (built from `crates/fastled-cli`, bundled into the wheel via `[tool.maturin] data`). Python `cli.py` / `app.py` are 15-line shims that call `fastled._rust_cli.invoke_rust_fastled_cli`.
- `fastled._native` is the PyO3 extension compiled from `crates/fastled-py/src/lib.rs`. It exposes Rust functions to Python (sketch discovery, build service, project init, string diff, frontend bundling, …).
- The compile path goes Rust → Python toolchain → Rust: `crates/fastled-cli/src/build.rs` constructs `fastled._native.NativeBuildService` and calls it; the Python toolchain (Meson/Ninja/Emscripten internals under `src/fastled/toolchain/`) is intentionally Python-owned.
- For PyO3 changes, after `cargo build -p fastled-py` the new `.dll` lands in `target/maturin/_native.dll`. `maturin develop` copies that to `src/fastled/_native.pyd`. If `_native.pyd` is locked by other Python processes (common on Windows), use the move-aside trick: `mv src/fastled/_native.pyd src/fastled/_native.pyd.old && cp target/maturin/_native.dll src/fastled/_native.pyd`. Remember to delete the `.pyd.old` before committing.

## Conventions

- Lint command: `bash lint` — runs cargo fmt + clippy, then ruff/black/isort/pyright. Must be green before commit.
- Test command: `bash test` (or targeted subset; see above).
- Cross-platform pyright warnings about `sys.platform` are expected — don't chase them.
- Always fix all diagnostics found, even pre-existing ones not caused by current changes, before declaring a change complete.
- Prefer native (Rust) implementations and fail-loud over silent Python fallbacks. The migration tracked in #14/#71/#72/#73 is moving everything in this direction.
- Don't add backwards-compat hacks for code paths that no longer exist (e.g. unused `_var` renames, "removed" comments).

## Common gotchas

- The installed `fastled` CLI on PATH (e.g. `/c/tools/python13/Scripts/fastled.exe`) may be stale; `find_rust_fastled_cli` in `src/fastled/_rust_cli.py` prefers `target/{release,debug}/fastled.exe` over PATH for exactly this reason. If a CLI test fails, check `uv run python -c "from fastled._rust_cli import find_rust_fastled_cli; print(find_rust_fastled_cli())"`.
- Error messages from the Rust CLI go to **stderr**. Tests asserting `result.stdout` for error text should use `result.stdout + result.stderr` (the parse_args.py removal in #72 exposed this).
- `tasklist` on Windows may report tens of thousands of `python.exe` zombies — most are stale tasklist artifacts, not real processes. `ps -ef | grep python` is more reliable for live ones.

## Filing issues / PRs

- `gh issue create --repo zackees/fastled-wasm …`
- `gh pr create …`
- Reference the originating issue (`Refs #72`) in commits/PRs when applicable.
1 change: 1 addition & 0 deletions CODEX.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
READ CLAUDE.md
1 change: 0 additions & 1 deletion crates/fastled-cli/src/archive.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
//! Archive download, verification, and extraction utilities.
//!
//! Ports the core logic from `emscripten_archive.py` and `frontend_esbuild.py`:
//! * HTTP(S) download with streaming
//! * SHA-256 file verification
//! * `.tar.zst` extraction (used for the emscripten toolchain)
Expand Down
166 changes: 105 additions & 61 deletions crates/fastled-cli/src/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@
//! The Rust CLI now drives the Python build service in-process through PyO3
//! instead of shelling out through `python -m fastled.app`.

use std::fs;
use std::path::{Path, PathBuf};
use std::time::Instant;

use anyhow::{Context, Result};
use pyo3::exceptions::PyKeyboardInterrupt;
use pyo3::prelude::*;
use pyo3::types::{PyDict, PyModule};

Expand Down Expand Up @@ -100,22 +102,16 @@ fn workspace_python_source() -> Option<PathBuf> {
src_dir.is_dir().then_some(src_dir)
}

fn py_path(py: Python<'_>, path: &Path) -> PyResult<Py<PyAny>> {
let pathlib = PyModule::import(py, "pathlib")?;
let cls = pathlib.getattr("Path")?;
Ok(cls
.call1((path.to_string_lossy().as_ref(),))?
.into_any()
.unbind())
fn normalize_path(path: &Path) -> PathBuf {
fs::canonicalize(path).unwrap_or_else(|_| path.to_path_buf())
}

fn py_fspath(value: &Bound<'_, PyAny>) -> PyResult<PathBuf> {
let py = value.py();
let os = PyModule::import(py, "os")?;
let fspath = os.getattr("fspath")?;
let path_value = fspath.call1((value,))?;
let path_str: String = path_value.extract()?;
Ok(PathBuf::from(path_str))
/// Canonicalize a sketch/`fastled_path` so the toolchain key registered with
/// `NativeBuildService.register_toolchain` matches the key passed into
/// `NativeBuildService.build` — mirrors `_toolchain_key` in
/// `src/fastled/build_service.py`.
fn toolchain_key(fastled_path: Option<&Path>) -> Option<String> {
fastled_path.map(|p| normalize_path(p).to_string_lossy().into_owned())
}

fn ensure_workspace_src_on_sys_path(py: Python<'_>) -> PyResult<()> {
Expand All @@ -140,80 +136,114 @@ fn run_build_embedded(request: &BuildRequest) -> Result<BuildResult> {
ensure_workspace_src_on_sys_path(py)
.context("failed to add workspace Python sources to sys.path")?;

let build_service_mod = PyModule::import(py, "fastled.build_service")
.context("import fastled.build_service")?;
let build_types_mod =
PyModule::import(py, "fastled.build_types").context("import fastled.build_types")?;
let native_mod =
PyModule::import(py, "fastled._native").context("import fastled._native")?;
let types_mod = PyModule::import(py, "fastled.types").context("import fastled.types")?;

let build_mode = types_mod
let build_mode_obj = types_mod
.getattr("BuildMode")
.context("BuildMode enum missing")?
.getattr(request.build_mode.py_member_name())
.with_context(|| {
format!("BuildMode.{} missing", request.build_mode.py_member_name())
})?;

let request_kwargs = PyDict::new(py);
request_kwargs
.set_item("sketch_dir", py_path(py, &request.sketch_dir)?)
.context("set sketch_dir")?;
request_kwargs
.set_item("build_mode", &build_mode)
.context("set build_mode")?;
request_kwargs
.set_item("profile", request.profile)
.context("set profile")?;
let service = native_mod
.getattr("NativeBuildService")
.context("NativeBuildService class missing")?
.call0()
.context("construct NativeBuildService")?;

// Register the Emscripten toolchain inline (still Python-owned).
let toolchain_mod = PyModule::import(py, "fastled.toolchain.emscripten")
.context("import fastled.toolchain.emscripten")?;
let toolchain_kwargs = PyDict::new(py);
match &request.fastled_path {
Some(path) => request_kwargs
.set_item("fastled_path", py_path(py, path)?)
.context("set fastled_path")?,
None => request_kwargs
Some(path) => toolchain_kwargs
.set_item("fastled_path", path.to_string_lossy().as_ref())
.context("set toolchain fastled_path")?,
None => toolchain_kwargs
.set_item("fastled_path", py.None())
.context("set fastled_path none")?,
.context("set toolchain fastled_path none")?,
}
request_kwargs
.set_item("force_clean", request.force_clean)
.context("set force_clean")?;

let build_request = build_types_mod
.getattr("BuildRequest")
.context("BuildRequest class missing")?
.call((), Some(&request_kwargs))
.context("construct BuildRequest")?;

let service = build_service_mod
.getattr("BuildService")
.context("BuildService class missing")?
.call0()
.context("construct BuildService")?;
let toolchain = toolchain_mod
.getattr("EmscriptenToolchain")
.context("EmscriptenToolchain class missing")?
.call((), Some(&toolchain_kwargs))
.context("construct EmscriptenToolchain")?;

let key = toolchain_key(request.fastled_path.as_deref());
service
.call_method1("register_toolchain", (toolchain, key.clone()))
.context("register_toolchain")?;

let sketch_dir_str = request.sketch_dir.to_string_lossy().into_owned();
let build_mode_str = request.build_mode.py_member_name();

let build_call = service.call_method1(
"build",
(
sketch_dir_str.as_str(),
build_mode_str,
build_mode_obj,
request.profile,
key.clone(),
request.force_clean,
),
);

let result = match build_call {
Ok(r) => r,
Err(err) => {
if err.is_instance_of::<PyKeyboardInterrupt>(py) {
// Mirror the Python wrapper: route through
// fastled.interrupts.handle_keyboard_interrupt so a
// worker-thread Ctrl+C wakes the main thread.
if let Ok(interrupts) = PyModule::import(py, "fastled.interrupts") {
if let Ok(handler) = interrupts.getattr("handle_keyboard_interrupt") {
let _ = handler.call1((err.clone_ref(py),));
}
}
}
return Err(err).context("NativeBuildService.build failed");
}
};

let result = service
.call_method1("build", (build_request,))
.context("BuildService.build failed")?;
let result_dict = result
.downcast_into::<PyDict>()
.map_err(|e| anyhow::anyhow!("NativeBuildService.build returned non-dict: {e}"))?;

let success: bool = result
.getattr("success")
let success: bool = result_dict
.get_item("success")
.context("missing build result success")?
.ok_or_else(|| anyhow::anyhow!("missing build result success"))?
.extract()
.context("extract build success")?;
let output: String = result
.getattr("stdout")
let output: String = result_dict
.get_item("stdout")
.context("missing build result stdout")?
.ok_or_else(|| anyhow::anyhow!("missing build result stdout"))?
.extract()
.context("extract build stdout")?;
let strategy: String = result
.getattr("strategy")
let strategy: String = result_dict
.get_item("strategy")
.context("missing build strategy")?
.ok_or_else(|| anyhow::anyhow!("missing build strategy"))?
.extract()
.context("extract build strategy")?;
let sketch_time_secs: f64 = result
.getattr("sketch_time")
let sketch_time_secs: f64 = result_dict
.get_item("sketch_time")
.context("missing sketch_time")?
.ok_or_else(|| anyhow::anyhow!("missing sketch_time"))?
.extract()
.context("extract sketch_time")?;
let output_dir = py_fspath(&result.getattr("output_dir").context("missing output_dir")?)
let output_dir_str: String = result_dict
.get_item("output_dir")
.context("missing output_dir")?
.ok_or_else(|| anyhow::anyhow!("missing output_dir"))?
.extract()
.context("extract output_dir")?;
let output_dir = PathBuf::from(output_dir_str);

Ok(BuildResult {
success,
Expand Down Expand Up @@ -407,6 +437,20 @@ mod tests {
assert!(src_dir.is_dir());
}

#[test]
fn test_toolchain_key_is_none_when_unset() {
assert_eq!(toolchain_key(None), None);
}

#[test]
fn test_toolchain_key_returns_some_when_set() {
// The canonicalize call may fail for nonexistent paths and fall back
// to the input path, which is fine for this test.
let p = PathBuf::from("/tmp/fastled-test-keypath-does-not-exist");
let key = toolchain_key(Some(&p));
assert!(key.is_some());
}

// ------------------------------------------------------------------
// BuildResult
// ------------------------------------------------------------------
Expand Down
Loading
Loading