Skip to content

Hosted requirements.txt rewrite replaces a user's own direct reference (six @ https://mirror/…/six-1.16.0-….whl, file:// fork) with the Socket PyPI build, and rollback then restores six==1.16.0 from PyPI, losing the original source #542

Description

[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).

Summary

scan --mode hosted treats any PEP 508 direct reference in requirements.txt whose wheel filename matches the patched name-version as a rewritable pin. The host doesn't matter: a private mirror, an internal fork build, or a file:// path all qualify. It overwrites the reference with the Socket-hosted artifact, which is built from the public PyPI release. The only warning is redirect_pypi_stale_install. Nothing says the user's chosen source was discarded.

rollback / remove then "restore" the line as six==1.16.0, a plain index pin. So after one scan → rollback round trip, the user's original source is permanently gone, and the next pip install -r pulls the public PyPI wheel instead of their fork or mirror.

Every other rewriter refuses a user-authored foreign origin:

  • bun: "A user's own URL dep (different origin, same leaf) is never claimed", crates/socket-patch-core/src/patch/redirect/mod.rs:8193
  • Pipenv: "a foreign file/path source … refuses the whole dependency" (redirect_pipenv_refused, CLI_CONTRACT.md)
  • Poetry: "a user-authored [package.source] on another origin" is refused (redirect_poetry_lock_unsupported)
  • cargo: a foreign registry pin refuses the dep
  • Vendored requirements.txt on the same file also leaves the line alone.

Impact

  • Silent code substitution. A project that pins an internally patched or forked wheel (same name and version, different bytes) gets the public release plus Socket's patch instead. Any fixes in the fork disappear without notice.
  • Source loss on unwind. rollback writes six==1.16.0, so a private-mirror or air-gapped project now resolves from the default index. For a file:// or mirror-only setup, that either fails or bypasses the mirror. This is similar to Hosted PDM rollback and remove replace a private index's static_urls with files.pythonhosted.org, so PDM bypasses the mirror (or fails on 2.12 behind a firewall) #413 (PDM static_urls), but here the hosted rewrite itself takes over the source.
  • The patch's beforeHash is computed from the PyPI bytes, so it doesn't even describe the fork the user actually had installed. Agent mode would refuse to apply to those bytes, but hosted swaps them out without asking.

Repro (real pip 24.0 / CPython 3.12, local mock patch API)

# a "private fork" of six 1.16.0 served from an internal host
mkdir private && <repack six-1.16.0-py2.py3-none-any.whl with an extra line "# INTERNAL-FORK-BUILD" in six.py>
(cd private && python3 -m http.server 18766 &)

mkdir proj && cd proj
printf 'idna==3.7\nsix @ http://127.0.0.1:18766/six-1.16.0-py2.py3-none-any.whl\n' > requirements.txt
python3.12 -m venv .venv && .venv/bin/pip install --no-deps -r requirements.txt   # installs the fork

socket-patch scan --mode hosted --org test-org --api-url <mock> --ecosystems pypi -y --json
#  redirect.redirected = 1, rewrittenFiles = [requirements.txt]
#  warnings: only redirect_pypi_stale_install
cat requirements.txt
#  idna==3.7
#  six @ <mock>/patch/pypi/six/1.16.0/<tok>/<uuid>/six-1.16.0-py2.py3-none-any.whl#sha256=…

socket-patch rollback --org test-org --api-url <mock> --patch-server-url <mock> -y
#  "Restored pkg:pypi/six@1.16.0 to its upstream registry entry"
cat requirements.txt
#  idna==3.7
#  six==1.16.0          <- the private URL is gone
pip install --no-deps -r requirements.txt   # now installs public PyPI six, not the fork

The same happens with six @ file:///abs/path/private/six-1.16.0-py2.py3-none-any.whl.

Expected vs actual

  • Expected: a direct reference to a non-Socket, non-PyPI origin is the user's own source choice. Hosted mode should refuse it with a warning, as the bun, Pipenv, Poetry and cargo rewriters do ("Version/source ambiguity is refused", docs/ecosystems.md PyPI row), and leave the line unchanged.
  • Actual: the line is rewritten (redirected: 1), there's no source-conflict warning, and rollback turns it into an index pin.

(Rewriting a files.pythonhosted.org URL, which is the same bytes as the index release, is arguably fine. Even then, rollback should restore the original URL rather than six==1.16.0.)

Matrix

OS pip / Python @ http://mirror @ file:// fork bare URL (no name @)
Linux pip 24.0 / CPython 3.12, main 61cfb9b rewritten (bug) rewritten (bug) not rewritten (redirect_requirements_entry_not_found)
Linux v4.0.0 (PyPI) rewritten (with --hash=, the pre-#383 shape) – –

Not a regression: v4.0.0 already does this. The logic is pure text rewriting with no platform branches, so macOS and Windows behave the same.

Suspect code

  • crates/socket-patch-core/src/patch/redirect/requirements.rs:138 (archive_version) accepts any URL whose filename parses as name-version, with no origin check.
  • crates/socket-patch-core/src/patch/redirect/requirements.rs:165-167 (requirement_version) turns an @ <location> tail into RequirementVersion::Exact, which makes the line rewritable.
  • The upstream restore (patch/redirect/upstream/pypi.rs) always writes name==version.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:triagedbugSomething isn't workingbughuntFound by a scheduled package-manager bug-hunt agentpm:pippip / requirements.txtpriority:p1

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions