Skip to content

Vendored Hatch wiring makes every Hatch install fail when the checkout path contains %XX, # or ; (e.g. a Jenkins "feature%2Fx" workspace), because Hatch's {root:uri} only escapes spaces #547

Description

[agent] Found by the scheduled Hatch bug-hunt routine (ledger #314).

Summary

Vendored Hatch wiring writes six @ {root:uri}/.socket/vendor/pypi/<uuid>/<wheel>#sha256=… into project.dependencies and env dependencies / extra-dependencies. docs/testing/hatch.md says this is done "so checkouts remain relocatable". But Hatch/Hatchling expand {root:uri} with hatchling/utils/fs.py:path_to_uri, which only escapes spaces:

def path_to_uri(path: str) -> str:
    if os.sep == "/":
        return f"file://{os.path.abspath(path).replace(' ', '%20')}"
    return f"file:///{os.path.abspath(path).replace(' ', '%20').replace(os.sep, '/')}"

So when the checkout path contains a URL-significant character, the expanded requirement points somewhere else, or doesn't parse at all:

  • %2F / %20 / any %XX: pip percent-decodes it, giving …/app_feature/six-patch/.socket/… and OSError: [Errno 2] No such file or directory.
  • #: the rest of the path becomes a URL fragment, giving No such file or directory: '/…/a'.
  • ;: Invalid requirement: 'six @ file:///…/semi': Expected a marker variable or quoted string.

Jenkins multibranch pipelines put the branch name into the workspace directory with / encoded as %2F (e.g. app_feature%2Fsix-patch), so this is a realistic CI path. The native (unwired) project installs fine in the same directory. Only the socket-patch wiring breaks it.

The scan itself gives no warning, even when it runs from such a path. It reports applied / success.

Impact

  • In any checkout whose absolute path contains %XX, # or ;, every Hatch env that carries a vendored reference fails to create (hatch run, hatch env create, hatch test). That includes the default env, because project dependencies go through the same expansion during the dev-mode install. CI goes red, and the cause is far from obvious.
  • vex in that checkout still emits not_affected / inline_mitigations_already_exist (from the committed artifact, with only a "live tree does not match" warning), even though no Hatch env can be installed there.
  • Hosted mode is unaffected (http URL).

This is the same class as the Maven #350 (file://${project.basedir} not URI-encoded), but in the Hatch lane.

Repro (Linux, Hatch 1.18.1; local mock patch API serving a patched six 1.16.0 wheel, --vendor-source service)

mkdir -p dev/app/src/app && cd dev/app && touch src/app/__init__.py && git init -q
cat > pyproject.toml <<'EOF'
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "app"
version = "0.1.0"
dependencies = ["six==1.16.0"]
[tool.hatch.build.targets.wheel]
packages = ["src/app"]
[tool.hatch.envs.lint]
extra-dependencies = ["six==1.16.0"]
EOF
socket-patch scan --vendor --vendor-source service --json     # applied, success
git add -A && git commit -qm vendored
git clone -q . "../../ws/app_feature%2Fsix-patch" && cd "../../ws/app_feature%2Fsix-patch"
hatch run python -c 'import six'
#   ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory:
#   '/…/ws/app_feature/six-patch/.socket/vendor/pypi/<uuid>/six-1.16.0-py2.py3-none-any.whl'
hatch run lint:python -c 'import six'                          # same error
git clone -q . ../plain && cd ../plain && hatch run python -c 'import six; print(six.SOCKET_PATCHED)'   # control: True

The same failure happens with the scan run directly inside directories named a#b, p%20q and semi;colon. Directories named at@sign and quo'te work.

Expected vs actual

  • Expected: docs/testing/hatch.md: "Vendored references use {root:uri} so checkouts remain relocatable." A vendored scan should either produce wiring that installs from any checkout path, or refuse / warn at scan time when the root contains characters that Hatch's {root:uri} doesn't encode (%, #, ;, ?). It should also document that a checkout can't be relocated to such a path.
  • Actual: the scan succeeds silently, and every Hatch env install fails in such checkouts.

OS × Hatch matrix (main 61cfb9b, vendored)

1.2.1 1.7.0 1.16.5 1.18.1
Linux, %2F path (default env + env extra-dependencies) fail fail fail fail (reproduced 2×)
Linux, # / %20 / ; path — — — fail
Linux, @ / ' / space path — pass (space, earlier run) — pass
Linux, same commit in a plain path (control) pass pass pass pass
macOS / Windows untested (same path_to_uri; probe branches currently can't be cleaned up by the routine)
Hosted, any path not affected

First bad

Present since vendored Hatch landed (#244). Released v4.0.0 has no Hatch lane.

Suspect code

  • crates/socket-patch-core/src/vendor/pypi_hatch.rs:46, :95, :180: they emit {root:uri}/… without checking whether the project root is URI-safe for Hatch's expansion.
  • crates/socket-patch-core/src/utils/hatch.rs:361 (vendored preflight): it checks the Hatch version but not the root path.
  • The root cause is upstream in hatchling/utils/fs.py:path_to_uri (only ' ' is escaped). A scan-time refusal or warning, plus a documented limitation, would avoid silently breaking CI.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions