Skip to content

Rebuild gem native extensions in agent-mode apply (#1996) - #1998

Merged
Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
fix/gem-native-ext-agent-apply
Nov 3, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
fix/gem-native-ext-agent-apply

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1996.

Problem

Agent-mode apply rewrote a gem's ext/**/*.c in place and reported the gem as patched. Ruby never loads those sources: it loads the extension RubyGems compiled at install time, so the vulnerable code kept running (e.g. every json 2.10.x–2.17.x patch for GHSA-x2f5-4prf-w687).

Fix: rebuild (the issue's option 2), with refusal as the fallback

New core module patch/gem_ext.rs, called from the gem branch of apply, from rollback, and from the installed-tree verdict in vex::verify.

  • Detection follows the issue: changed .c .h .cc .cpp .cxx .hh .hpp .hxx .m .y .rl, extconf.rb, ext/**/depend, or any file under the directory of a build script the installed gemspec's extensions names (this covers Rust/CMake/Rakefile extensions). A .y/.rl file in a gem that declares no extensions is not an input, since racc/ragel output ships pre-generated.
  • Rebuild: the patched sources are written, then RubyGems' own Gem::Ext::Builder#build_extensions runs (what gem pristine --extensions runs). The Ruby used is the one that owns the install: <prefix>/bin/ruby for a <prefix>/lib/ruby/gems/<abi> home (never a prefix inside the project), else ruby on PATH, run from --cwd so rbenv/asdf pick the project's Ruby. A candidate is accepted only if its spec resolves to this gem dir and its extension dir is the installed one. The child runs without RUBYOPT/RUBYLIB/BUNDLE*, with the gem home first on GEM_PATH.
  • Old build moved aside first: ExtConfBuilder never overwrites files already in the extension dir, so an in-place build would keep the stale .so. The first rebuild also backs up the install-time build (the extension dir plus its lib/ copies) to <gem_dir>/.socket-patch-native-orig/.
  • Marker: <gem_dir>/.socket-patch-native-build.json records the source hashes the build used and the hash of every artifact, including the lib/ copies. In testing, Ruby actually loaded the lib/ copy, not the one under extensions/.
  • Failure: a build that fails puts the previous build and the original sources back: native_extension_build_failed, with the tail of the compiler output.
  • Refusal: when the copy can't be rebuilt at all (no specifications/ entry, a precompiled platform gem, no matching Ruby), it is refused with native_extensions_unsupported and nothing is written. This also happens in --dry-run.
  • Already patched: sources an earlier release patched without rebuilding get rebuilt, with a native_extension_rebuilt warning. A copy whose marker already matches is already_patched and nothing is rebuilt.
  • VEX / apply --check: an installed gem copy with native inputs is attested only while its marker matches the record and every listed artifact. Otherwise it fails with native_extension_not_rebuilt. Hosted copies are unchanged.
  • Rollback: after the sources are restored, the backup is put back, which needs no compiler. Without a backup, it rebuilds from the restored sources. If that fails: native_extension_restore_failed, with a gem pristine hint.

Codes are documented in CLI_CONTRACT.md (new "Ruby gems with native extensions in agent mode" section), and docs/ecosystems.md gets a "Ruby gems: native extensions" section.

Testing

  • New tests/e2e_gem_native_ext_build.rs builds a real C-extension gem (SpNative.value goes from 1 to 2):
    • apply → Ruby loads 2 → --check clean → re-apply skipped → rollback → 1, marker and backup gone
    • sources patched without a rebuild → --check fails with native_extension_not_rebuilt → apply rebuilds
    • a patch that doesn't compile → everything put back, byte-identical
    • Hermetic, no Ruby needed: no Ruby on PATH → refused (dry run too); gem without extensions → refused; Ruby-only patch to a native gem → applies without Ruby; stale build → --check fails.
  • The two compile capstones are #[ignore]d like the other Ruby suites and added to the Ruby 3.4 PR leg and the Ruby 3.1 e2e-full leg (SOCKET_PATCH_BUNDLER_E2E_REQUIRED makes a missing toolchain fail there).
  • Unit tests cover gemspec parsing, classification, marker verification and backup restore.
  • Manual check on the real json 2.10.2 gem (two extensions) on macOS/Ruby 3.4: patched parser.c → Ruby raises the patched error message, apply --check is clean (also after moving the project dir), and rollback restores the original message.
  • Local: core lib 7126 passed, CLI lib 868 passed, plus e2e_gem, in_process_gem_apply, e2e_redirect_gem_stale_install, e2e_vex_redirect, help_text_hygiene and cli all pass. cargo fmt is clean, and clippy is clean on every touched file.

depscan and the patches harness

  • depscan production paths are unaffected. Hosted serving (depscan#29554) and autopatch use hosted mode only, never agent-mode apply.
  • The patches evaluation harness (Patchpedia) runs agent-mode apply on gems (apply --manifest-path … --global --global-prefix $GEM_HOME/gems --offline --force), then recompiles the extension itself, with era-specific fallbacks: DISABLE_SSL, an era OpenSSL, nokogiri port-archive restore. Its worker builds socket-patch from depscan's pinned submodule, so this change reaches it only on a pin bump.
  • New --no-gem-native-rebuild / SOCKET_NO_GEM_NATIVE_REBUILD lets a caller that compiles the extension itself keep the old behavior: sources are written, no Ruby is run, and the run warns native_extension_not_rebuilt. Patchpedia can set the env var before the bump; the currently pinned binary ignores it.
  • Replayed the harness flow in Debian bookworm with ruby-full (Ruby 3.1.2 / RubyGems 3.3.15, what the eval workers run) on the real json 2.10.2 gem, using a Linux build of this branch:
    • Default mode: apply alone makes the patched C code run; the harness recompile and the gem-shim re-apply on top still work; rollback restores the original.
    • Opt-out mode: sources are written, the old build stays loaded until the harness recompiles, then the patched code runs.
  • Older Rubies:
    • The exact probe/build script also works on Ruby 2.1.10.
    • The script requires rubygems itself, since Ruby 1.8 doesn't load it and RUBYOPT is cleared.
    • RubyGems older than 2.2 (the harness's Ruby 1.8.2 rung, RubyGems 0.9.4) is refused by name.

Code review

/code-review high reported 10 findings. 9 are fixed in 73a7d0b:

  • A planted backup could make rollback delete outside the gem. Only the gem's own extension dir is now replaced; regression test included.
  • C files outside ext/ in a gem with no extensions are no longer refused.
  • A root-level build script no longer makes every file a native input.
  • A failed build-record write is now fully undone, so the error message is true.
  • The Ruby/ABI check is no longer vacuous.
  • Loaded libraries are no longer rewritten in place on restore.
  • A build timeout now kills the whole process group.
  • Stray probe output is tolerated.
  • Duplicated helpers removed.

Not fixed: "apply now runs the gem's build scripts". That's inherent to rebuilding, and it's what gem install / bundle install already ran for that project.

Known limits

  • A stale extension dir left by another Ruby ABI in the same gem home isn't rebuilt. RubyGems won't load a gem whose extension isn't built for the running Ruby, so it's normally unused.
  • The build timeout is 30 min, since nokogiri's vendored libxml2 build takes minutes.
  • A precompiled platform gem can't be fixed in agent mode, and hosted mode refuses it too (redirect_gem_platform_unsupported). The refusal message points at the ruby-platform gem.

🤖 Generated with Claude Code


Note

Medium Risk
Agent-mode apply now invokes Ruby and the gem's own build scripts against installed trees, which expands blast radius if rebuild or restore logic is wrong; attestation rules also change for native gem patches.

Overview
Agent-mode apply no longer stops at rewriting a gem's ext/ sources. When a patch touches native-extension build inputs, it now rebuilds the extension with RubyGems' builder (same path as gem pristine --extensions) so Ruby loads patched compiled code, not stale .so artifacts.

The flow refuses installs that can't be rebuilt in place (Bundler git checkouts, precompiled platform gems, no matching Ruby), backs up the install-time build under .socket-patch-native-orig/, and writes .socket-patch-native-build.json so vex and apply --check only attest copies whose built artifacts match the patched sources. Failed builds roll sources and the old build back; rollback restores the backup (or rebuilds when needed). Callers that compile themselves can use --no-gem-native-rebuild / SOCKET_NO_GEM_NATIVE_REBUILD to keep source-only writes with a native_extension_not_rebuilt warning.

Docs (CLI_CONTRACT.md, docs/ecosystems.md) and CI add the e2e_gem_native_ext_build capstone on pinned Ruby matrix legs.

Reviewed by Cursor Bugbot for commit 73a7d0b. Configure here.

Agent-mode apply rewrote a gem's ext/**/*.c in place and reported it
patched, while Ruby kept loading the extension compiled at install time.

When a gem record changes a native-extension build input, apply now
writes the patched sources and rebuilds the extension with RubyGems' own
builder (the step `gem pristine --extensions` runs), using the Ruby that
owns the install. The old build is moved aside first (ExtConfBuilder
never overwrites files already in the extension dir) and backed up once
inside the gem dir; a marker records the source hashes it was built from
and every artifact's hash, including the lib/ copies Ruby may load.

- A failed build puts the sources and the previous build back:
  native_extension_build_failed.
- A copy that cannot be rebuilt (precompiled platform gem, no
  specifications entry, no matching Ruby) is refused with
  native_extensions_unsupported and nothing is written.
- Sources an earlier release patched without a rebuild are rebuilt
  (native_extension_rebuilt warning).
- vex / apply --check attest only when the marker matches
  (native_extension_not_rebuilt otherwise); hosted copies are unaffected.
- rollback restores the backed-up original build, or rebuilds from the
  restored sources without one (native_extension_restore_failed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mikolalysenko Mikola Lysenko (mikolalysenko) added the pm:bundler Bundler (RubyGems) label Nov 3, 2026
Review fixes:
- rollback restores only the gem's own extensions/<plat>/<abi>/<gem>
  dir under its home; a backup or marker naming any other path (or a
  symlink) is refused before anything is deleted.
- In a gem that declares no extensions, only C sources under ext/ are
  native inputs (templates and fixtures elsewhere stay plain files); a
  root-level build script makes only top-level files inputs.
- A build whose record cannot be written is undone like a failed one.
- The Ruby must build into an installed extension dir; a gem without
  one (RubyGems < 2.2) is refused.
- Lib copies are restored by stage + rename, never rewritten in place
  under a process that has them mapped.
- A build timeout kills the whole process group (Unix).
- Probe output tolerates stray stdout lines; the script requires
  rubygems itself (Ruby 1.8 with RUBYOPT cleared) and names a RubyGems
  older than 2.2.
- Share gemspec_declares_extensions and copy_tree_blocking.

--no-gem-native-rebuild (SOCKET_NO_GEM_NATIVE_REBUILD) is for a caller
that compiles the extension itself, such as the patches evaluation
harness: sources are written without a rebuild, with a
native_extension_not_rebuilt warning, and vex does not attest them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review November 3, 2026 19:45

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix is ON, but none of the reported issues met your autofix severity threshold.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 73a7d0b. Configure here.

- CI (child_deadline_guard): the Ruby probe/build no longer keeps its
  own kill_on_drop deadline. New utils::process::build_output_within is
  the bounded spawn for builds: stdout and stderr captured, the child in
  its own process group (Unix) killed whole at the deadline.
- Bugbot: the lib-copy snapshot, the previous marker, backup.json and
  the backed-up lib copies are read through the regular-file guard, so a
  FIFO in the gem dir fails fast instead of hanging apply or rollback.
- Bugbot: a dry run over sources that are already patched but whose
  extension was not built from them warns native_extension_rebuilt
  ("apply would rebuild it") instead of reading as plain already_patched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mikolalysenko
Mikola Lysenko (mikolalysenko) added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 7bde3ba Nov 3, 2026
50 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the fix/gem-native-ext-agent-apply branch November 3, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pm:bundler Bundler (RubyGems)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent-mode apply reports gem native-extension patches as applied without rebuilding

2 participants