Repository navigation
chore: move all HTTP downloads from Python to Rust; drop httpx (closes #63) - #64
Merged
Merged
Conversation
The Rust CLI prints its listening URL to stdout once the socket is bound. spawn_http_server() already parses that URL via regex and uses it. The follow-up HTTP probe through httpx duplicated that gate without adding information; the fallback path keeps a 1s sleep for the case where the URL line wasn't seen in time. Removes one httpx callsite.
The Python compile path used to download and extract the emscripten
toolchain itself (httpx + pyzstd, via toolchain/emscripten_archive.py).
That now lives in Rust:
- crates/fastled-cli/src/install.rs (new) - ensure_emscripten_installed()
fetches the platform manifest, downloads multipart parts with SHA-256
verification, extracts the .tar.zst, and writes the .emscripten config
into ~/.fastled/toolchains/emscripten/{platform}/{arch}/{version}/. Uses
ctcb-manifest (workspace dep) for the manifest schema and the existing
archive.rs primitives for download/sha256/extract/config.
- crates/fastled-cli/src/main.rs - compile_and_serve() now calls
ensure_emscripten_installed() once, before invoking Python, and exports
FASTLED_EMSCRIPTEN_DIR for the child process to consume.
- crates/fastled-cli/src/archive.rs - dropped #![allow(dead_code)] since
the module is now wired into the compile flow.
Python:
- src/fastled/toolchain/emscripten.py - _get_clang_tool_chain_emscripten_dir()
reads FASTLED_EMSCRIPTEN_DIR first; ensure_clang_tool_chain_emscripten()
is now a thin lookup (the install is upstream). The clang_tool_chain
PyPI fallback was removed.
- src/fastled/toolchain/emscripten_archive.py - deleted (Rust owns it).
- pyproject.toml - dropped clang-tool-chain runtime dep.
Net: Python runtime deps now httpx, fasteners, zcmds_win32 (Win-only).
Lint clean, 118 unit tests pass.
…#59) The Python frontend bundling step used to download esbuild from npmjs itself (httpx + tarfile + fasteners InterProcessLock). That now lives in Rust: - crates/fastled-cli/src/install.rs - ensure_esbuild_installed() downloads the npm registry .tgz, extracts the single bin into ~/.fastled/toolchains/esbuild/{platform}/{arch}/{version}/, returns the path. - crates/fastled-cli/src/archive.rs - new extract_member_from_tgz() helper plucks a single file out of a gzipped tarball (needs flate2, added to workspace). - crates/fastled-cli/src/main.rs - compile_and_serve() now also calls ensure_esbuild_installed() and exports FASTLED_ESBUILD_PATH for the Python subprocess. Python: - src/fastled/frontend_esbuild.py - install_esbuild() removed; the module now resolves esbuild via FASTLED_ESBUILD_PATH (set by Rust) or shutil.which fallback. Drops httpx, tarfile, fasteners imports from this file. - src/fastled/toolchain/emscripten_archive.py was the only other fasteners consumer (already gone in the previous commit), so fasteners drops from pyproject.toml entirely. Net: runtime Python deps now httpx, zcmds_win32 (Win-only). Lint clean, 118 unit tests pass.
Python's project_init.py used httpx to query the GitHub API for the latest
release tag and to download the FastLED repo zip. That logic now lives in
Rust:
- crates/fastled-cli/src/install.rs - ensure_fastled_repo(ref: Option<&str>)
- latest-release lookup via api.github.com (User-Agent header set)
- resolve_fastled_ref: handles commit SHAs, tags, branches; HEAD-checks
a tag URL before falling back to a branch URL (mirrors Python)
- download via archive::download, extract via archive::extract_zip
- cache at ~/.fastled/cache/fastled-{ref}/ (cheap library.json check on
subsequent calls)
- crates/fastled-cli/src/main.rs - two integration points:
- --init pre-step: when --init is set, Rust calls ensure_fastled_repo
(deriving the ref from --latest / --commit / --branch) and exports
FASTLED_LOCAL_REPO_DIR before delegating to Python.
- --internal-ensure-fastled-repo [REF] hidden flag: prints the local
path to stdout for the Python-driven Api.project_init / Api.get_examples
paths to consume via subprocess.
Python:
- src/fastled/project_init.py rewritten to use either FASTLED_LOCAL_REPO_DIR
(preferred) or the Rust binary's --internal-ensure-fastled-repo flag.
Removes the httpx, tempfile-based extract, and DownloadResult helpers
that used to live here. _find_rust_fastled_cli() walks the workspace
target/ tree so it returns the Rust binary, not the Python entry-point
shim that shadows it in .venv/Scripts/.
Lint clean. test_api.py::test_project_init now exercises the Rust path
end-to-end and passes (21s).
Last remaining Python httpx callsites moved into the Rust CLI:
- crates/fastled-cli/src/install.rs - ensure_chrome_extension(id, name):
GET the Chrome Web Store CRX with the proper User-Agent / Referer the
endpoint expects, scan the body for the PK\x03\x04 (or PK\x05\x06)
ZIP magic, strip the CRX header, extract the embedded zip into
~/.fastled/chrome-extensions/{name}/. Cached via manifest.json check.
- crates/fastled-cli/src/main.rs - new hidden plumbing flag
--internal-ensure-chrome-extension EXTENSION_ID NAME that prints the
install path to stdout (mirrors the FastLED-repo flag).
- src/fastled/playwright/chrome_extension_downloader.py - thin shim that
subprocesses the Rust binary. ChromeExtensionDownloader class deleted
(no external callers).
- src/fastled/toolchain/emscripten.py - EmscriptenToolchain._download_fastled
used httpx to grab FastLED master for native compile when no local repo
was specified. Now delegates to project_init._ensure_repo_via_rust (the
Rust binary handles the download) and copies the cached repo into the
short-name target_dir/repo layout the build pipeline expects.
pyproject.toml: drop httpx. Net runtime deps now: zcmds_win32 (Win-only).
Lint clean, 119 unit tests pass, no `import httpx` left in src/.
This was referenced May 13, 2026
zackees
added a commit
that referenced
this pull request
May 13, 2026
Sweeps the non-Python items issue #59 flagged that are still on main after PRs #61, #62, #64, #65, #66, #68 cleaned up the Python side: * `entrypoint.sh` — Docker container entrypoint. Sourced `/emsdk/emsdk_env.sh` (container path) and called `fastled_wasm_compiler.cli_update_from_master` (a module from a separate package that only exists inside the container). * `compiler/run.py` — Docker compile-server entry point. Imports `fastled_wasm_compiler`, a separate package not in this repo. * `compiler/debug.sh` — instructed user to run from `/js` (container working directory). Empty `compiler/` directory removed. * `.dockerignore` — only contained cache exclusions (`**/.mypy_cache/`, `**/.ruff_cache/`) which belong in `.gitignore` if anywhere; no Docker build context this could serve. * `chrome_vscode_bridge_design_task.md` — one-off design draft that opens with "Here's a refined ... file for your repo. Feel free to adjust...". Zero references anywhere else. * `upload_package.sh` — manual PyPI upload helper, superseded by the automated `publish_release.yml` workflow. Zero inbound references. * Empty root-level `fastled/` directory left over from a prior layout. `lint` script updated to stop passing the (now-deleted) `compiler/` path to ruff / black / isort / pyright / the keyboard-interrupt checker. `bash lint` clean. `bash test` — 119 pass. Verified by grep across `*.py`, `*.rs`, `*.toml`, `*.yml`, `*.sh`, `*.md`, `Cargo.toml` (excluding `.venv/` and `.claude/worktrees/`): none of the deleted files are referenced from a live code path or workflow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Continues from PR #62. Moves the four remaining httpx callsites in Python into Rust install primitives, then drops httpx from
pyproject.toml. Python runtime deps: 2 → 1 (zcmds_win32, Windows-only).Closes #63.
Commits
chore(open_browser): drop redundant wait_for_server HTTP probe— the Rust server prints its listening URL on stdout; the follow-up httpx GET added nothing.chore(rust): own emscripten install in the Rust CLI—install::ensure_emscripten_installed()does manifest fetch + multipart SHA-256 +.tar.zstextract +.emscriptenconfig. Wired intocompile_and_serve; exportsFASTLED_EMSCRIPTEN_DIR. Uses thectcb-manifestcrate. Dropsclang-tool-chainandpyzstd.chore(rust): own esbuild install in the Rust CLI; drop fasteners—install::ensure_esbuild_installed()downloads the npm tarball and plucks the bin via the newarchive::extract_member_from_tgz. ExportsFASTLED_ESBUILD_PATH.chore(rust): own FastLED repo download in the Rust CLI—install::ensure_fastled_repo(ref)handles latest-release lookup, commit/tag/branch resolution, zip download, and extraction to~/.fastled/cache/fastled-{ref}/. Two integration points:--initpre-step (setsFASTLED_LOCAL_REPO_DIRbefore Python takes over).--internal-ensure-fastled-repo [REF]flag for direct Python-API callers (Api.project_init / Api.get_examples) — prints the local path and exits.Python's
project_init.pywas rewritten around this;EmscriptenToolchain._download_fastledalso routes through it.chore(rust): own Chrome extension CRX download; drop httpx—install::ensure_chrome_extension(id, name)downloads the CRX with the picky User-Agent / Referer headers, scans for the PK magic, strips the CRX header, extracts the embedded zip. Hidden--internal-ensure-chrome-extensionflag for the Python playwright shim.Dep change
pyproject.toml)httpx,fasteners,clang-tool-chain,zcmds_win32)zcmds_win32; sys_platform == 'win32')pyzstd(via emscripten_archive.py)ctcb-manifest = "0.4.1",flate2,dirsgrep -r 'import httpx' src/returns nothing.Test plan
bash lint— cargo check + clippy + rustfmt + ruff + black + isort + pyright all greenbash test— 119 passed, 1 skipped, 0 failures (was 118;Api.project_inittest now exercises the Rust path end-to-end in 21s)~/.fastled/toolchains/emscripten/...~/.fastled/toolchains/esbuild/...--init wasm— empty~/.fastled/cache/, expect Rust to fetch FastLED + Python to scaffold~/.fastled/chrome-extensions/, expect Rust to download + extractCloses #63
Refs #59
🤖 Generated with Claude Code