Skip to content

feat(toolchain): install a whole pinned ESP-IDF release as one fbuild operation - #1981

Merged
zackees merged 1 commit into
mainfrom
agent/esp-idf-release-install
Nov 10, 2026
Merged

zackees merged 1 commit into
mainfrom
agent/esp-idf-release-install

Conversation

@zackees

@zackees zackees commented Nov 10, 2026

Copy link
Copy Markdown
Member

Part 5 (last piece) of #1970 — follow-up 1 of the #1761 research programme.

The previous four PRs pinned the pieces: tool archives (#1975), the install
profile + IdfEnvironment (#1976), the source tree (#1978) and the Python
environment (#1980). Nothing installed the release. This adds the one entry
point that does, and returns the environment a delegated idf.py build (#1971)
must hand its subprocess.

What

crates/fbuild-toolchain/src/toolchain/esp_idf_install.rs

  • EspIdfInstall::new(project_dir, profile) → install() installs
    source → tools → python, then returns an InstalledEspIdf.
  • InstalledEspIdf is the answer to "what do I run idf.py with?":
    • path_dirs() — the venv bin first, then the tool bin directories in
      install order. Ordering is not cosmetic: idf.py's shebang resolves
      python from PATH, so a host interpreter answering first fails ESP-IDF's
      own dependency check on a correct environment.
    • env_vars() — IDF_PATH, IDF_PYTHON_ENV_PATH, ESP_ROM_ELF_DIR,
      OPENOCD_SCRIPTS. No IDF_TOOLS_PATH: that variable points
      idf_tools.py back at an ~/.espressif-shaped side tree, which is exactly
      what feat(toolchain): pin an ESP-IDF release as fbuild packages with an esp32s3-only install profile #1970 removes.
    • missing_units() names a partial tree per unit rather than letting it read
      as a suspiciously small byte count.
    • installed_bytes() sums the three cache entries du-style (hardlinks
      counted once, one identity set per entry).
  • IdfInstallUnit (Source / Tools / Python) exists because the three
    units are independent downloads with independent keys: a re-run after a failed
    pip step must not refetch the 450 MB source tree.

A unit already installed is skipped by its own package, so re-running is a no-op
and a partial failure is repaired without re-downloading what landed.

Real output (ignored e2e, actually run)

soldr cargo test -p fbuild-toolchain --lib -- --ignored esp_idf_release_install

test toolchain::esp_idf_install::esp_idf_install_tests::esp_idf_release_install_provisions_the_whole_release ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 204 filtered out; finished in 190.12s
v5.5.1: tools 1149.2 MB, release total 1684.7 MB

That single test installs the whole release into one cache root (24 source
archives, 4 tool archives, the pinned CPython + hashed wheels: ~450 MB of
downloads, ~1.6 GB on disk) and then asserts the combination, not the
pieces:

  • ESP-IDF's own tools/check_python_dependencies.py, executed out of the
    installed source tree by the installed interpreter with the installed
    constraints/requirements, prints Python requirements are satisfied. — the
    same gate idf.py runs at startup.
  • The installed xtensa-esp32-elf-gcc reports esp-14.2.0_20241119.
  • Tools ≤ 1.3 GB (feat(toolchain): pin an ESP-IDF release as fbuild packages with an esp32s3-only install profile #1970 AC2, measured, currently 1149.2 MB); release total
    1684.7 MB.
  • Second install of the whole release: sentinel file survives and the measured
    tree changes only by the sentinel's own bytes — nothing re-downloaded or
    re-extracted (AC4).

Gates

  • soldr cargo test -p fbuild-toolchain --lib → 198 passed; 0 failed; 6 ignored
  • soldr clippy --workspace --all-targets -- -D warnings → clean, 0 warnings
  • soldr cargo fmt --all -- --check → no diffs
  • enforce_platform_boundary.py --print-totals → rows=44; dylint_rows=43, rc 0
  • check_test_layout.py → ok: 8 integration-test categories, rc 0
  • check_dylint_allowlists.py → Dylint allowlist paths are current., rc 0
  • No new workspace crate, no board VID/PID literals.

Deliberately not in this PR

… operation

esp_idf_install.rs composes the three packages #1970 already pins — source tree,
tool archives and Python environment — into EspIdfInstall::install(), which
returns the environment a delegated idf.py build (#1971) must hand its
subprocess: the venv bin first on PATH, IDF_PATH, IDF_PYTHON_ENV_PATH,
ESP_ROM_ELF_DIR and OPENOCD_SCRIPTS, and deliberately no IDF_TOOLS_PATH.

InstalledEspIdf names missing units instead of reporting a small byte count,
and installed_bytes() sums the three entries du-style.

Real e2e (ignored, run): 1 passed in 190.12 s — 24 source archives, 4 tool
archives and the pinned CPython all installed into one cache root, then
ESP-IDF's own tools/check_python_dependencies.py ran out of the installed
tree with the installed interpreter and printed 'Python requirements are
satisfied.', xtensa gcc reported esp-14.2.0_20241119, tools measured 1149.2 MB
(<= 1.3 GB) and the release total 1684.7 MB; a second install of the whole
release is a no-op.

Refs #1970
@coderabbitai

coderabbitai Bot commented Nov 10, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

1 participant