Fix lock-only requirements.txt discovery (#412, #523)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #412
Fixes #523

Summary

A pip project with no venv yet (a fresh clone, a CI job or a Docker build) is discovered through the lockfile-only requirements.txt inventory. That inventory missed two common ways of writing pins, so those packages never reached the patch API. scan printed "No patches available" and exited 0 in both hosted and vendored mode, and pip install -r requirements.txt then installed the unpatched release. With this PR, discovery reads the same pins pip installs.

Root cause

inventory_requirements_txt in vendor/lock_inventory/pypi.rs read requirements differently from both pip and socket-patch's own writers:

• Lockfile-only scan ignores pins in requirements.txt -r includes, so a fresh checkout reports "No patches available" and installs unpatched (hosted and vendored) #412: it read only the root requirements.txt and skipped -r lines without following them. The vendored writer, by contrast, already walks every in-root include.
• Lockfile-only requirements.txt discovery skips exact pins written with spaces (six == 1.15.0, six (==1.15.0)), so a fresh checkout reports "No patches available" and pip installs the unpatched release #523: it recognized pins with utils::requirements::exact_pin, which only looked at the first whitespace token. So six == 1.15.0, six ==1.15.0, six== 1.15.0, six[x] == 1.15.0 and the legacy six (==1.15.0) weren't seen as pins. pip and the hosted rewriter (requirement_version) both accept all of these.

Changes

• utils::requirements::exact_pin now parses the requirement the way pip does: name, then optional extras, optional parentheses and ==, with whitespace allowed between them. Only options (--hash=…) may follow the version. As a side effect, a range like six==1.0,<2 is no longer mistaken for the pin 1.0,<2. Lock-only vex discovery (vex::discover::pypi_other) uses the same function and gets the fix too.
• vendor::pypi_requirements::requirements_includes is the include parser pulled out of the vendored writer's walk, now shared. is_in_root_rel is now pub(crate).
• inventory_requirements_txt reads the root file plus every in-root -r / --requirement include through a new requirements_tree walk. The walk runs over the ProjectView, so the in-memory hosted engine reads the same tree. It resolves each include relative to the including file, guards against cycles, never follows -c constraints, and skips out-of-root or absolute includes, matching the writer. The public_index rule (an index option makes hashed pins unverifiable) now spans the whole tree, because pip applies options from any file globally.

In hosted mode, an installed pin that lives only in an include is documented as rewrite-refused (redirect_requirements_entry_not_found). Lock-only checkouts now reach that same documented path instead of silently skipping the package.

Tests (red → green)

Issue	Test	Without fix	With fix
#523	utils::requirements::tests::exact_pin_is_the_shared_registry_pin_rule (spaced, tab, paren and extras forms; ==1.*, ,<2 and stray-paren negatives)	FAILED	ok
#523	vendor::lock_inventory::tests::requirements_spaced_and_parenthesised_pins_are_inventoried	FAILED	ok
#523	scan_requirements_lock_only::lock_only_scan_discovers_spaced_pins (built binary; hosted + vendored; purls must reach /patches/batch)	FAILED	ok
#412	vendor::lock_inventory::tests::requirements_in_root_includes_are_inventoried (nested and relative includes, --requirement=, -rfile, cycle, -c and ../ not followed, ProjectView::Memory)	FAILED	ok
#412	vendor::lock_inventory::tests::requirements_index_option_in_an_include_spans_the_tree	FAILED	ok
#412	scan_requirements_lock_only::lock_only_scan_discovers_included_pins (built binary; hosted + vendored)	FAILED	ok

Validation

• cargo clippy --workspace --all-features -- -D warnings: clean.
• cargo test --workspace --all-features --no-fail-fast: 214 targets. 12 tests fail only because this sandbox runs as root, which ignores the read-only or unremovable fixtures they rely on (*_write_failure_*, *unremovable*, relax_loop_must_not_traverse_symlinked_root). All 12 pass when re-run as a non-root user. Everything else passes.
• cargo test -p socket-patch-cli --test e2e_vex_build -- pip:: --ignored (real pip, every major, hosted + vendored → manifest-less VEX): ok.
• cargo fmt --all -- --check is not clean on main itself with the pinned 1.93.1 toolchain (128 files), and CI doesn't run it. The files this PR touches are rustfmt-clean, and no unrelated files were reformatted.
• The npm, pypi and gem wrappers are unaffected: the change is entirely in the Rust core.
• CI on acb0abb: 477 success, 6 skipped (matrix templates and canary/downgrade), 0 failed. native (ubuntu-latest, 2.9.3) failed once in its PDM extras vendored lane (appliedExactlyOne). That lane reads pdm.lock and never reaches the requirements.txt inventory, and it passed on its single re-run.
• Cursor Bugbot on acb0abb: no issues found.

Follow-ups

None.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JASDoVJNrZUBy9kd7fWPxN

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit acb0abb. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Ready for review at acb0abb.

• CI: all required checks green on acb0abb (latest run per check; no failures, Bugbot neutral).
• Mergeable against main, no conflicts.
• Cursor Bugbot reviewed acb0abb: no issues found; no unresolved review threads.
• Reviewer focus: the requirements.txt lock-only inventory now follows -r includes and matches pins the way pip / socket-patch's writers do (Lockfile-only scan ignores pins in requirements.txt -r includes, so a fresh checkout reports "No patches available" and installs unpatched (hosted and vendored) #412, Lockfile-only requirements.txt discovery skips exact pins written with spaces (six == 1.15.0, six (==1.15.0)), so a fresh checkout reports "No patches available" and pip installs the unpatched release #523).

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Reviewed acb0abb5f4130b8d3ef52a48dcb6c2a8a53fa4f2. Recommendation: ready to merge as-is from code review.

No actionable correctness/security regressions found. Checked pin parsing, relative includes/cycle guards, in-memory parity, global index-option handling, and the shared writer/VEX behavior.

Validation: cargo test -p socket-patch-core --lib requirements — 87 passed; cargo test -p socket-patch-cli --test scan_requirements_lock_only — 2 passed, exercising hosted and vendored discovery through the binary. Full workspace/platform matrix not rerun.