Skip to content

feat(toolchain): pin ESP-IDF v5.5.1 tool components as fbuild packages - #1975

Merged
zackees merged 2 commits into
mainfrom
agent/esp-idf-release-pins
Nov 9, 2026
Merged

zackees merged 2 commits into
mainfrom
agent/esp-idf-release-pins

Conversation

@zackees

@zackees zackees commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Part of #1970 (follow-up 1, tool components slice).

Why

The merged research pass (docs/design/bare-idf-component-builds.md, #1969) measured IDF
v5.5.1's own bootstrap: install.sh needs a venv with a working ensurepip, and
openocd-esp32's install check loads libusb-1.0.so.0, so it cannot provision a toolchain on
every host fbuild supports (it fails on this repo's own NixOS host both ways). A delegated
idf.py build (#1971) therefore needs the toolchain to arrive as ordinary fbuild packages.

What

crates/fbuild-toolchain/src/toolchain/esp_idf_release.rs — pins the archives Espressif
publishes for ESP-IDF v5.5.1 (tools/tools.json, status: recommended, commit
fcae3288) and installs them through the existing PackageBase pipeline:

  • IdfComponent / IdfProfile::Esp32S3Only: xtensa-esp-elf, esp32ulp-elf, esp-rom-elfs,
    openocd-esp32 — and nothing else. riscv32-esp-elf (2.1 GiB) and xtensa-esp-elf-gdb
    (104 MiB) are not in the pin table at all, so no code path can install them for an S3 project.
  • EspIdfTool installs through PackageBase: download → sha256 verify → extract → .install_complete; it exposes an inherent ensure_normalized() returning NormalizedPath rather than a Package impl, because Package::ensure_installed is typed PathBuf and ban_std_pathbuf keeps that legacy signature allowlisted (no consumer needs the trait object yet)
    sentinel, then returns the tool root (flat archives and name-version/name/ nesting both work).
  • Cache key is esp-idf-v5.5.1/<component> and the install leaf is
    v5.5.1+<component version>, so two IDF releases install side by side and neither entry is
    shared/overwritten (the setup action: fbuild cache restore-keys fallback crosses fbuild versions and doubles the saved cache #1843/fix(ci): scope the setup action's fbuild caches to one fbuild content hash (#1843) #1844 cache-key rule).
  • platform_key maps fbuild's hosts to tools.json spellings (linux-amd64, linux-arm64,
    macos, macos-arm64, win32); linux-armel/armhf/i686 are deliberately unpinned and
    return an error instead of borrowing another host's binaries. esp-rom-elfs falls back to
    the release's host-independent any archive.

Measured, not estimated (du -sb, linux-amd64, install.sh-installed v5.5.1 tree):

component installed
xtensa-esp-elf 1,103,937,348 B
esp32ulp-elf 31,188,493 B
esp-rom-elfs 6,918,628 B
openocd-esp32 7,153,401 B
ESP32-S3 profile 1,149,197,870 B (~1.07 GiB)

vs riscv32-esp-elf 2,142,092,152 B + xtensa-esp-elf-gdb 108,817,092 B that the default
installer adds (3.4 GiB of tools, ~3.8 GiB with the dist tarballs).

Verification (real output)

$ soldr cargo test -p fbuild-toolchain --lib esp_idf_release
test result: ok. 8 passed; 0 failed; 1 ignored; 0 measured; 162 filtered out

$ soldr cargo test -p fbuild-toolchain --lib esp_rom_elfs_installs -- --ignored
test toolchain::esp_idf_release::tests::esp_rom_elfs_installs_from_the_pinned_archive ... ok
test result: ok. 1 passed; 0 failed; 0 ignored   (0.94 s)

The ignored test is the end-to-end proof: it downloads the pinned esp-rom-elfs archive into an
empty cache via EspIdfTool::ensure_installed, verifies its sha256, extracts
esp32s3_rev0_rom.elf, then calls ensure_installed again and gets the same tree from cache (no
re-download — the offline-second-install shape of the acceptance criterion).

Pin spot-check against the live object:

$ curl -sSL .../esp-rom-elfs-20241011.tar.gz | sha256sum
921f000164a421c7628fbfee55b173384aafaa51883adc65cd27bf9b0af9e9a9  (== pin)

Gates: soldr clippy -p fbuild-toolchain --all-targets -- -D warnings clean;
ci/check_test_layout.py ok; ci/enforce_platform_boundary.py --print-totals still 44/43
(no new host-gated rows).

Not in this PR

  • No IDF source checkout and no Python environment — feat(espidf): opt-in delegating idf.py build producing <build>/idf/ artifacts #1971 needs them to run idf.py; the
    source is a 603 MB checkout per version and needs its own pin/provisioning decision.
  • No idf.py invocation, no integration=idf-cmake emission, no change to the
    Arduino-on-IDF path. The esp32s3-only profile installs <= 1.3 GB acceptance criterion is
    measured and asserted in a test, but from the recorded du of an install.sh tree, not
    from a full fbuild-package install of all four components (that is the AC1 run, ~1.15 GB of
    downloads, and belongs with the follow-up that actually builds through the toolchain).
  • No new workspace crate, no board VID/PID literals.

install.sh cannot provision ESP-IDF on every host fbuild supports: it needs
a venv with a working ensurepip, and openocd-esp32's install check loads
libusb-1.0.so.0. Pin the archives the release publishes instead (tools.json,
status recommended) and install them through PackageBase.

- IdfComponent/IdfProfile::Esp32S3Only: xtensa-esp-elf, esp32ulp-elf,
  esp-rom-elfs, openocd-esp32; riscv32-esp-elf (2.1 GiB) and xtensa-esp-elf-gdb
  are absent from the pin table.
- Cache key esp-idf-v5.5.1/<component>, leaf v5.5.1+<component version>: two
  IDF releases install side by side (#1843 rule).
- Unpinned hosts (linux-armel/armhf/i686) error instead of borrowing another
  host's binaries; esp-rom-elfs uses the release's `any` archive.
- Measured S3 profile: 1,149,197,870 B installed (du -sb), asserted in a test.

Part of #1970 (tool components slice; source checkout + python env remain for
the idf.py follow-up).
@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

`dylints/ban_std_pathbuf` denied three signatures in the new module:
`install_path`, `ensure_normalized` and `tool_root` all spelled
`std::path::PathBuf`, which is exactly the unnormalized-path class the
lint exists to prevent (#436, #826). The new file is not
and must not be on the shrink-only allowlist.

The public surface now returns `fbuild_core::path::NormalizedPath`, and
`tool_root` returns `Result<NormalizedPath>` instead of a path that may
not exist -- so a caller cannot "install" into a directory the release
never produced. `is_installed` now also requires the exported directory,
so a cache hit that lost its tree is repaired rather than trusted.

Dropping `impl Package` is deliberate: `Package::ensure_installed` is
typed `PathBuf` in fbuild-packages-fetch (an allowlisted legacy file),
and no consumer of this type needs the trait object yet. `is_installed`
+ `get_info` stay as inherent methods.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

1 participant