Fix vendored Pipenv hash export breaking pip (#981)

crates/socket-patch-cli/CLI_CONTRACT.md

@@ -947,7 +947,7 @@ to **six flavors**.
 | pypi / uv (uv.lock) | rebuilt wheel (canonical PEP 427 filename; RECORD regenerated) | `[tool.uv.sources] <name> = {path}` in pyproject + surgical uv.lock rewrite; transitive deps via `[tool.uv] override-dependencies` | `uv sync --locked` / `--frozen --offline` (hash-verified, byte-stable lock) |
 | pypi / poetry (poetry.lock: legacy `[metadata.hashes]`, lock 1.0/1.1 `[metadata.files]`, 2.x `files`) | (rebuilt wheel) | lock-only: the target `[[package]]` gets `[package.source] type="file"` (+ `reference = ""` on the 0.12/1.0 layouts, which read it unconditionally) and the single `{file, hash: sha256-of-our-wheel}` entry in whichever table the generation keeps it. pyproject + `metadata.content-hash` untouched; CRLF locks keep their line endings. A lock written by Poetry < 1.4 emits `pypi_poetry_integrity_unverified` (that installer verifies no local hashes and skips an already-installed version) | `poetry check --lock && poetry sync`, cold cache (hash fail-closed from Poetry 1.4; byte-stable lock) — see `docs/testing/poetry-compatibility.md` |
 | pypi / pdm (pdm.lock) | (rebuilt wheel) | lock-only: the `[[package]]` gains the local-file `path` + `files[]` hash. pyproject + `content_hash` untouched. Non-fixture `[metadata] strategy` / hash-less locks refused | `pdm sync` (+ `pdm install --check`), cold cache |
-| pypi / pipenv (Pipfile.lock) | (rebuilt wheel) | lock-only: the `default`/`develop` entry → `{file, hashes:[sha256-of-our-wheel]}`. Pipfile + `_meta.hash` untouched. Emits `vendor_integrity_unverified` — pipenv does not hash-check file entries; the committed wheel bytes are the protection | `pipenv install --deploy` (+ `pipenv verify`), cold cache |
+| pypi / pipenv (Pipfile.lock) | (rebuilt wheel) | lock-only: the `default`/`develop` entry → `{file, hashes:[sha256-of-our-wheel]}`. Pipfile + `_meta.hash` untouched. Emits `vendor_integrity_unverified` — pipenv does not hash-check file entries; the committed wheel bytes are the protection — and, when another entry is hashed, `pypi_pipenv_hash_export_unsupported` (`pipenv requirements --hash` drops the vendored wheel's hash) | `pipenv install --deploy` (+ `pipenv verify`), cold cache |
 | pypi / requirements.txt (pip / `uv pip`) | (rebuilt wheel) | pin line → `./<wheel>` (markers carried over; transitive deps appended), plus `--hash=sha256:<hex>` only when the requirements tree is already in pip's hash-checking mode (any `--hash` or `--require-hashes`) | `pip install -r` / `uv pip install -r` **run from the project root** (both resolve bare paths against the CWD) |
 | nuget | deterministically rebuilt `.nupkg` at `<idLower>.<versionNorm>.nupkg` (the uuid dir IS a NuGet folder feed; the stale embedded signature is dropped — unsigned is accepted under NuGet's default validation), plus `<uuid>/.gitignore` (`!*`: re-includes the nupkg against the project's ignores, such as VisualStudio.gitignore's `*.nupkg`); refuses `vendor_artifact_gitignored` when git would still drop it | `nuget.config` source + `packageSourceMapping` for the id (creating the mapping from scratch ALSO fans a `<package pattern="*" />` out to every pre-existing source, including the ones NuGet inherits from the user config and parent directories' configs (v5.0 #354), none at all when an inherited config already maps packages — mapping is exclusive, NU1100 otherwise; another source's exact pattern for the id is set aside in a comment while vendored, `vendor_nuget_mapping_set_aside`, #462) **+** `packages.lock.json` `contentHash` → `base64(sha512(nupkg))` for the entries at the patched version in every lock a project under the root restores into — the root `packages.lock.json`, member projects' locks, `packages.<Project>.lock.json`, a literal `NuGetLockFilePath` (v5.0 #353/#514; an unevaluable `NuGetLockFilePath` is refused with `vendor_nuget_lock_path_unresolved`) — (`vendor_nuget_no_lockfile` warning when there is none; a lock that also resolves the id at another version is refused with `vendor_nuget_lock_other_version`, nothing written) | `dotnet restore --locked-mode`, cold cache, `--network none` (tampered nupkg fails NU1403) |
 | maven | the patched `.jar` + the upstream pom (only its `<version>` suffixed; transitives survive) + `.sha1` sidecars + an ownership marker under `.socket/vendor/maven2/<g-path>/<a>/<v>-socket.<hex8>/`; every JVM tree root (`.socket/vendor/maven2`, Gradle's `.socket/vendor/gradle`, Coursier's `.socket/vendor/coursier`) owns a `.gitignore` (`!*`) that re-includes the jars against the project's ignores, such as Java.gitignore's `*.jar`, plus a `.gitattributes` (`-text`); a tree root git ignores itself refuses `vendor_artifact_gitignored` | every pom root, single-module (a reactor of one) or multi-module: the pinned `<version>` + `<dependencyManagement>` pin, `.mvn/maven.config` (`maven.repo.local.tail`) and the `socket-patch-vendor` fallback file repository (`checksumPolicy=fail`); Gradle, sbt and scala-cli roots go to the same JVM backend (ledger ecosystem `jvm`). Pre-v5 `maven_pom_repository` entries (`<repository>` to `.socket/vendor/maven/<uuid>`) are revert-only: vendoring their root is refused (`vendor_jvm_shape_unsupported`, `legacy_maven_root`) | `mvn` build on a fresh checkout with a warm local repository and behind `mirrorOf external:*` (host capstone `e2e_vendor_maven_build` across the Maven matrix) |
@@ -1605,6 +1605,7 @@ command's integration tests check its output.
 | `vendor_yarn_berry_mixed_line_endings` | `failed` | vendor (yarn berry): `yarn.lock` or the root `package.json` mixes CRLF and LF line endings (or holds a bare CR) — no single ending can be kept, and yarn rewrites such a file wholesale on its next install (a mixed lock also fails `--immutable`, YN0028). Refused before any write; `yarn install` normalizes the files. A uniformly CRLF pair is vendored in CRLF. A hosted→vendored takeover (`vendor`, `scan`/`get --mode vendored`) raises this — and the berry `vendor_yarn_berry_cache_unsupported` gates — BEFORE restoring the hosted pin's upstream entry (dry run too), so a refused purl stays hosted. |
 | `vendor_override_conflict` | `failed`        | vendor (pnpm/yarn-berry): a user-authored override/resolution for the package already exists. pnpm exception: a user exact pin equal to the vendored version, under the bare `<name>` or the `<name>@<version>` key, in package.json `pnpm.overrides` or pnpm-workspace.yaml `overrides:` (#854; there a quoted `'1.3.0'` or a trailing `# comment` is the same pin), is taken over in place (its value becomes the vendored `file:` spec under the user's own key on every override surface) and `vendor --revert` restores it byte-for-byte. A range, another version, a `>` selector, several same-name keys, or package.json and pnpm-workspace.yaml pinning under different keys still refuse; the detail names the file that carries the override. |
 | `vendor_integrity_unverified` | `skipped` (warning) | vendor (pipenv): the lockfile format does not hash-check file entries; the committed wheel bytes are the protection. |
+| `pypi_pipenv_hash_export_unsupported` | `skipped` (warning) | vendor / scan / get `--mode vendored` (pipenv, #981), on a run that wires the lock and on an in-sync re-run, unless the probed Pipenv is a pre-2023 release (2022 exports the wheel with its hash): another `Pipfile.lock` entry carries `hashes`, and `pipenv requirements --hash` (Pipenv 2023.7.9+) exports the vendored `file` entry with no `--hash` while those entries keep theirs, so `pip install -r` of the export refuses the whole install in hash-checking mode. No lock shape avoids it. The package is still vendored; remedy: export without `--hash`, or `--mode hosted` (its `#sha256=` URL exports a hash pip accepts). |
 | `vendor_content_mismatch_overwritten` | — | **No longer emitted** (service-only vendoring): `vendor` commits the patch server's verified artifact without reading the installed bytes, so it has no mismatch to report. Kept so consumers that match on it know it will not appear. |
 | `vendor_vlt_transitive_unsupported` | `failed` | vendor (vlt): the target has an inbound edge from another package in `vlt-lock.json` (the detail names it); vendored mode rewires only direct dependencies of the root or a workspace member, because vlt silently reverts transitive lock surgery. Remedy: `--mode hosted`. Refused before any download or write, dry runs included (`would_refuse`). |
 | `vendor_vlt_lock_out_of_sync` | `failed` | vendor (vlt): an importer's `package.json` is missing, unparseable, or declares a spec for the dependency that differs from the lock's importer edge. Remedy: `vlt install` first. Refused before any write. |

crates/socket-patch-core/src/vendor/pypi.rs

@@ -1308,6 +1308,13 @@ async fn pypi_prelude<'p>(
             if target == PipenvTarget::Fresh {
                 warnings.extend(project.warnings.iter().cloned());
             }
+            // Like the stale install below, the export gap (#981) outlives
+            // the run that wired the lock: an in-sync re-run warns too.
+            warnings.extend(super::pypi_pipenv::hash_export_warning(
+                &project,
+                &canon_name,
+                installer,
+            ));
             // Both a fresh vendor and a re-run over an already-wired lock
             // keep warning while the venv still holds the upstream release.
             if let Some(stale) = pipenv_stale_install_warning(
@@ -7259,6 +7266,87 @@ wheels = [
         );
     }
 
+    /// #981: a lock with another hashed registry entry exports through
+    /// `pipenv requirements --hash` with hashes on every line but the
+    /// vendored wheel's, which pip's hash-checking mode refuses. The vendor
+    /// run warns, and so does an in-sync re-run; a lock that pins only the
+    /// target exports no hash and does not, nor does a Pipenv 2022 project,
+    /// whose export keeps the wheel's hash.
+    #[tokio::test]
+    async fn pipenv_vendor_warns_the_hash_export_gap() {
+        async fn vendor_with(fx: &E2eFixture, major: u32) -> Vec<VendorWarning> {
+            const PURL: &str = "pkg:pypi/six@1.16.0";
+            let sources = PatchSources::blobs_only(&fx.blobs);
+            let service = crate::vendor::test_support::service_fixture::Fixture::new(
+                PURL,
+                fx.site_packages.as_path().into(),
+                &fx.record,
+                &sources,
+            )
+            .await;
+            let outcome = vendor_pypi_with_pipenv_version(
+                PURL,
+                &fx.site_packages,
+                &fx.root,
+                &fx.record,
+                &sources,
+                "2026-06-09T00:00:00Z",
+                false,
+                false,
+                Some(&service.cfg),
+                &tokio::sync::OnceCell::new_with(Some(Some(major))),
+                &InstalledSiteListings::default(),
+            )
+            .await;
+            let VendorOutcome::Done {
+                warnings, entry, ..
+            } = outcome
+            else {
+                panic!("vendor did not complete: {outcome:?}");
+            };
+            if let Some(entry) = entry {
+                save_ledger_entry(&fx.root, &entry).await;
+            }
+            warnings
+        }
+        let warns = |warnings: &[VendorWarning]| {
+            warnings
+                .iter()
+                .any(|w| w.code == "pypi_pipenv_hash_export_unsupported")
+        };
+
+        let fx = pipenv_e2e_fixture().await;
+        let warnings = vendor_with(&fx, 2026).await;
+        assert!(!warns(&warnings), "{warnings:?}");
+
+        let hashed_develop = PIPENV_REGISTRY_LOCK.replace(
+            r#""develop": {}"#,
+            r#""develop": {
+        "idna": {
+            "hashes": ["sha256:bbb"],
+            "index": "pypi",
+            "version": "==3.7"
+        }
+    }"#,
+        );
+        let fx = pipenv_e2e_fixture().await;
+        touch(&fx.root, "Pipfile.lock", &hashed_develop).await;
+        let warnings = vendor_with(&fx, 2026).await;
+        assert!(warns(&warnings), "{warnings:?}");
+        let lock = tokio::fs::read_to_string(fx.root.join("Pipfile.lock"))
+            .await
+            .unwrap();
+        assert!(lock.contains(UUID), "Pipfile.lock not wired:\n{lock}");
+        // The in-sync re-run keeps warning: the export gap is still there.
+        let warnings = vendor_with(&fx, 2026).await;
+        assert!(warns(&warnings), "{warnings:?}");
+
+        let fx = pipenv_e2e_fixture().await;
+        touch(&fx.root, "Pipfile.lock", &hashed_develop).await;
+        let warnings = vendor_with(&fx, 2022).await;
+        assert!(!warns(&warnings), "{warnings:?}");
+    }
+
     const SIBLING_EXPORT: &str = "-i https://pypi.org/simple\nsix==1.16.0\n";
 
     /// Vendor with Pipfile.lock alone and persist the entry: a project

crates/socket-patch-core/src/vendor/pypi_pipenv.rs

@@ -150,6 +150,51 @@ pub(super) async fn load_pipenv_project(
     })
 }
 
+/// #981: Pipenv 2023.7.9 and later export a `file` / `path` entry without
+/// `--hash` (`pipenv requirements --hash`), while every hashed registry
+/// entry keeps its hashes. One hashed line turns on pip's hash-checking
+/// mode, which then refuses the whole install over the vendored wheel. No
+/// lock shape avoids it (a `#sha256=` fragment on the relative path breaks
+/// `pipenv sync`), so warn whenever an entry other than `canon_name`'s
+/// would export a hash, unless `installer` is a release before 2023 (2022
+/// exports `file:///<abs> --hash=…`; earlier ones have no `requirements`).
+/// Hosted mode exports `<url>#sha256=<hex>`, which pip accepts as the hash.
+pub(super) fn hash_export_warning(
+    p: &PipenvProject,
+    canon_name: &str,
+    installer: Option<u32>,
+) -> Option<VendorWarning> {
+    if installer.is_some_and(|major| major < 2023) {
+        return None;
+    }
+    let hashed = category_names(&p.lock).into_iter().find_map(|section| {
+        let entries = p.lock[&section].as_object()?;
+        entries
+            .iter()
+            .find(|(key, entry)| {
+                canonicalize_pypi_name(key) != canon_name
+                    && entry
+                        .get("hashes")
+                        .and_then(Value::as_array)
+                        .is_some_and(|hashes| !hashes.is_empty())
+                    && entry.get("file").is_none()
+                    && NON_REGISTRY_KEYS.iter().all(|k| entry.get(*k).is_none())
+            })
+            .map(|(key, _)| format!("{section}.{key}"))
+    })?;
+    Some(VendorWarning::new(
+        "pypi_pipenv_hash_export_unsupported",
+        format!(
+            "`pipenv requirements --hash` (Pipenv 2023.7.9 and later) exports the vendored \
+             {canon_name} wheel without a hash while hashed {LOCK_FILE} entries (such as \
+             {hashed}) keep theirs, so `pip install -r` of an export holding both refuses \
+             the whole install (\"Hashes are required in --require-hashes mode\"); export \
+             without `--hash`, or use `--mode hosted`, whose reference exports with a hash \
+             pip accepts"
+        ),
+    ))
+}
+
 /// Target-specific guards (also re-run by the wire step right before
 /// writing). Entries match by PEP 503 canonical NAME in every package
 /// category. Registry pins and existing vendored wheel identities
@@ -1267,6 +1312,63 @@ mod tests {
         assert_eq!(p.warnings[0].code, "vendor_integrity_unverified");
     }
 
+    /// #981: `pipenv requirements --hash` (2023.7.9+) exports the vendored
+    /// `file` entry without a hash while every hashed registry entry keeps
+    /// its `--hash`, so pip's hash-checking mode refuses the whole install.
+    /// A lock with another hashed registry entry warns.
+    #[tokio::test]
+    async fn hash_export_warning_when_other_entries_are_hashed() {
+        let tmp = write_lock(LOCK_TRANSITIVE_REGISTRY).await;
+        let p = load_pipenv_project(tmp.path()).await.unwrap();
+        let w = hash_export_warning(&p, "six", None).expect("python-dateutil is hashed");
+        assert_eq!(w.code, "pypi_pipenv_hash_export_unsupported");
+        assert!(
+            w.detail.contains("pipenv requirements --hash"),
+            "{}",
+            w.detail
+        );
+        assert!(w.detail.contains("--mode hosted"), "{}", w.detail);
+        assert!(w.detail.contains("vendored six wheel"), "{}", w.detail);
+        assert!(w.detail.contains("default.python-dateutil"), "{}", w.detail);
+        // A 2023+ installer warns; 2022 exports the wheel with its hash.
+        assert!(hash_export_warning(&p, "six", Some(2026)).is_some());
+        assert!(hash_export_warning(&p, "six", Some(2022)).is_none());
+
+        // A hashed entry in another category counts too: `--dev` and
+        // `--categories` exports carry it.
+        let develop = LOCK_DIRECT_REGISTRY.replace(
+            r#""develop": {}"#,
+            r#""develop": {
+        "idna": {
+            "hashes": [
+                "sha256:028ff3aadf0609c1fd278d8ea3089299412a7a8b9bd005dd08b9f8285bcb5cfc"
+            ],
+            "index": "pypi",
+            "version": "==3.7"
+        }
+    }"#,
+        );
+        let tmp = write_lock(&develop).await;
+        let p = load_pipenv_project(tmp.path()).await.unwrap();
+        assert!(hash_export_warning(&p, "six", None).is_some());
+    }
+
+    /// With no other hashed registry entry, the export has no `--hash`
+    /// line, pip never turns on hash-checking, and there is nothing to warn
+    /// about: the target alone, or beside a `file` reference.
+    #[tokio::test]
+    async fn hash_export_warning_absent_without_other_hashed_entries() {
+        let tmp = write_lock(LOCK_DIRECT_REGISTRY).await;
+        let p = load_pipenv_project(tmp.path()).await.unwrap();
+        assert!(hash_export_warning(&p, "six", None).is_none());
+
+        // python-dateutil's only sibling is six's vendored `file` entry,
+        // which the export writes without a hash.
+        let tmp = write_lock(LOCK_TRANSITIVE_VENDORED).await;
+        let p = load_pipenv_project(tmp.path()).await.unwrap();
+        assert!(hash_export_warning(&p, "python-dateutil", None).is_none());
+    }
+
     /// Our serializer reproduces pipenv's own
     /// `json.dumps(indent=4, sort_keys=True) + "\n"` byte-for-byte, so a
     /// parse → serialize round trip of a pipenv-written lock is the identity.