Repository navigation
feat(espidf): build an opted-in component project with the pinned IDF - #1984
Merged
Merged
Conversation
fbuild had no CMake and no idf.py, so a genuine idf_component_register() project could not be built at all (#1664) and has been rejected since #1730. Add the explicit opt-in that changes that, and only that: board_build.build_system = espidf (default `native`, so no existing project changes behaviour). The orchestrator branches right after the common BuildContext, so an opted-in environment never resolves the Arduino core, its archives or its linker recipe. Two supervised steps run through fbuild_core::subprocess: idf.py set-target <chip>, then idf.py build, both with -C <project> -B <build>/idf. Artifacts are discovered from IDF's own flasher_args.json rather than a second copy of IDF's layout, validated, published flat under <build>/idf/ and recorded in idf-artifacts.json as the proof follow-ups 3-5 consume. The generated sdkconfig lives in the project directory, not the build directory - found only by running the real build. A new additive core API, run_command_reporting, keeps both bounded streams when a deadline fires; run_command drops them with the cancelled capture future. DelegateFailure renders stage, argv, cwd, exit code, budget and both stream tails into the build error, so a delegated failure is never a bare exit code. Real e2e (ignored, tests/platform/espidf_component): 1 passed in 129.30s warm, 311 s cold with the release install; app image 208,528 B and sdkconfig 73,024 B match the #1761 spike byte for byte, 22,949,145 B of artifacts published. Closes #1971.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
The delegated-build branch pushed build.rs from 999 to 1017 lines, which the "Reject .rs files over 1000 LOC" job fails on a file that was legal at the base commit. Move the two guards into prepare_build_system (they belong beside reject_unsupported_esp_framework) and the disk pre-flight's guard + phase into disk_requirements::preflight_if_needed, which owns those numbers. Behaviour is unchanged: both run at exactly the same points, before package resolution. build.rs is 996 lines.
Dylint Full rejected two spellings in the new tests, both already recorded as pitfalls: - ban_std_pathbuf on a Vec<std::path::PathBuf> built from split_paths in the argv test; the entries are NormalizedPath now. - ban_raw_path_prefix_compare on Path::strip_prefix in the fixture copy; NormalizedPath::relative_to decides containment instead. 235 lib tests still pass.
This was referenced Nov 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
fbuild compiles every ESP32 environment with the Arduino core against
precompiled ESP-IDF
.alibraries: there is no CMake and noidf.py, so agenuine
idf_component_register()project cannot be built at all (#1664), andsince #1730 it is rejected instead of being silently mis-built. This is
follow-up 2 of the research programme in #1761 (
docs/design/bare-idf-component-builds.md,recommended option 3): keep the native Arduino path untouched and add an
explicit opt-in delegating path.
Depends on #1970, whose whole release (source, toolchains, CMake, Ninja, Python
environment) is already on
main.What
Opt-in.
board_build.build_system = espidf(defaultnative, so noexisting project changes behaviour). An unknown value is a config error naming
the valid ones. With the opt-in, a bare
framework = espidfis permitted; the#1730 rejection and its warning for
arduino, espidfare unchanged without it.The chip must be
esp32s3— the one target the pinned release provisions — andanother MCU is a configuration error by name, not a missing compiler later.
Delegation (
esp32/espidf_delegate.rs). After the commonBuildContext(so the board and build directory are resolved) the orchestrator branches
before any Arduino package/framework work:
PATHis the profile's entries prepended to the hostPATHandIDF_PATH/IDF_TOOLS_PATHpoint into the fbuild cache tree, both measuredrequirements from #1983.
Artifacts (
esp32/espidf_artifacts.rs). Discovery reads IDF's ownflasher_args.jsonrather than hard-coding IDF's layout, validates every file,publishes the two nested ones flat, and writes
idf-artifacts.json(name, path,byte count) as the artifact proof follow-ups 3–5 consume. All of
bin/elf/map/bootloader.bin/partition-table.bin/sdkconfig/compile_commands.jsonend up under<build>/idf/.Structured failures. A new additive core API,
fbuild_core::subprocess::run_command_reporting, spawns with a deadline butkeeps both streams:
run_commanddrops the output of a timed-out child withthe cancelled capture future, so "the build timed out, here is what it printed"
was not expressible. The streams are drained into shared buffers (so a
surviving grandchild costs the last bytes, never the report), bounded to a
64 KiB tail with the total byte count recorded, and the tree is stopped with the
existing containment.
DelegateFailurerenders stage, argv, cwd, exit code,budget and both bounded streams into the build error; a timeout is
FbuildError::Timeoutwith the same body.Non-goals (per the issue): no zccache integration, no compile-db re-rooting, no
multi-version IDF matrix, and no new integration-mode string — that lands with
the coverage gate in #1972.
Real output
E2E (ignored test, real install + real
idf.pyovertests/platform/espidf_component,a
maincomponent thatREQUIRES mycompplus a second component):E2E (ignored test, real install + real
idf.pyovertests/platform/espidf_component: amaincomponent thatREQUIRES mycompplus a second component):
soldr cargo test -p fbuild-build-esp --lib -- --ignored espidf_component_project→
test result: ok. 1 passed; 0 failed; 236 filtered out; finished in 129.30s(warm install). The first, cold run of the same build took 311 s including the
~1.8 GB release install.
Published under
<build>/idf/, real byte counts fromidf-artifacts.json:component_demo.bincomponent_demo.elfcomponent_demo.mapbootloader.binpartition-table.binsdkconfigcompile_commands.jsonThe build directory is 163 MB, the compile database has more than 100 entries,
and the app image (208,528 B) and
sdkconfig(73,024 B) match the #1761reference spike byte for byte.
One defect only a real build could find:
idf.pywrites the generatedsdkconfigin the project directory, next tosdkconfig.defaults, not inthe build directory, so the first artifact pass failed with "did not produce
the generated sdkconfig".
generated_sdkconfig()now reads<project>/sdkconfig(with the build-directory copy as a fallback) andpublishes it flat.
Unit gates:
fbuild-core --lib382 passed / 2 ignored,fbuild-build-esp --lib235 passed / 2 ignored, workspaceclippy --all-targets -- -D warningsclean,
cargo fmt --checkclean.Acceptance criteria
the_delegated_opt_in_permits_a_bare_espidf_project,bare_espidf_framework_is_rejected_before_building).native. Not re-measured on this host (loadavg 31/16, full pioarduino core build) — CI's existing ESP32 jobs are the measurement.a_wedged_step_times_out_with_both_streams,timeout_stops_the_child_and_keeps_what_it_printed).idf.py(argv, cwd, env, artifact discovery) plus one ignored end-to-end test.Closes #1971. Unblocks #1972, #1973, #1974.
Relationship to other work
Distinct from draft #1878 (which vendors IDF through
platformio/framework-espidfpackages for C5/P4): this path provisions one pinned release in fbuild's own
cache and delegates the project's own CMake build, which is what #1971 scoped.