Fix PDM settings ignoring PDM's config layers (#609, #566)

[Mikola Lysenko (mikolalysenko)]

Final-head CI is complete: 335 successful checks, 6 skipped; no failures or pending checks. Bugbot passed, there are no unresolved review threads, and the PR is mergeable.

Fixes #609 and #566. PDM environment discovery previously missed string-valued and user/site python.use_venv settings, so a project configured for __pypackages__ could select an activated or stray venv. Global discovery also missed site-level project, venv and interpreter locations.

The crawler now resolves project settings through PDM's layers: PDM_USE_VENV, legacy .pdm.toml overrides over pdm.toml, the selected user config, then site config. It accepts TOML booleans and the strings PDM writes, while preserving missing-key fallback. Global discovery also reads the site layer while retaining its existing user-directory inventory.

Site configuration follows the first absolute, trimmed XDG_CONFIG_DIRS entry on Linux and macOS. Empty and relative entries are skipped, and / remains a valid root. Linux and Windows retain their platform defaults.

On macOS, the implicit site directory comes from the Python runtime used by the installed PDM launcher. This distinguishes Homebrew's <prefix>/share/pdm from runtimes using /Library/Application Support/pdm. A bounded probe imports platformdirs directly without invoking PDM or its plugin loader. It uses a private working directory, null stdin, a five-second timeout, bounded output and child cleanup, and removes Python import/bootstrap environment overrides. Higher-priority project/user settings and explicit XDG paths bypass the probe.

Unknown launchers, a missing platformdirs, or a failed probe leave the implicit macOS site layer undiscovered. Ordinary HOME-based user-site installs are supported; installs available only through a custom PYTHONUSERBASE are not discovered by this probe. These limits are documented in docs/testing/pdm-compatibility.md.

Validation on 3226a8f2:

• All 77 focused Python crawler tests passed, including legacy-versus-modern precedence, environment priority, macOS project/global site settings, launcher parsing, process bounds, timeout cleanup and environment isolation. The precedence and site regressions have preserved failing-before/passing-after evidence.
• All 18 native runtime controls passed: six inherited-environment scenarios across uv and spaced-path Homebrew venvs containing PDM 2.29.2/platformdirs 4.12.2, plus a HOME-based Homebrew user-site platformdirs fixture. The selected directories matched each runtime; planted project/import code did not run and the parent environment remained unchanged.
• Native layer-resolution controls also passed with platformdirs 4.9.6 and 4.12.2. These checks exercised real PDM configuration and interpreter locations without repeating the full package/network compatibility matrix locally.
• Changed code passes formatting and diff checks. Targeted clippy passed with the existing macOS unused_variables warning allowed; three pre-existing formatting deviations were retained. The final Git objects match the tested files, two independent reviews found no remaining issue, and the commit merges cleanly with current main 045d7ec7.
• Full CI, compatibility workflows, benchmarks, and Bugbot completed successfully on the corrected commit 3226a8f2.

---

Note

Medium Risk
Changes which Python site-packages paths are selected for patching; macOS adds a subprocess probe with mitigations but still depends on launcher parsing and PATH.

Overview
Fixes PDM environment discovery so python.use_venv and global install paths follow PDM’s real config stack instead of only project pdm.toml / .pdm.toml.

python.use_venv is now resolved in order: PDM_USE_VENV, legacy .pdm.toml over pdm.toml, then the single user config file ($PDM_CONFIG_FILE or platformdirs’ chosen config.toml), then site config.toml. Values match PDM 2.27+ by accepting TOML bools and string forms like "false". That fixes PEP 582 (__pypackages__) vs stray .venv when users set use_venv in user or site config (#609).

Global PDM paths (global_project.path, venv.location, python.install_root) also merge in the site config layer, not just user config (#566). Site dirs use the first absolute $XDG_CONFIG_DIRS segment (with Linux/Windows defaults); user config no longer falls through multiple candidate directories.

On macOS, when XDG doesn’t pin site config, a new pdm_site module parses the pdm launcher’s shebang, runs a bounded platformdirs probe in that interpreter (no pdm CLI), with timeouts, output limits, and stripped PYTHON* env vars. Higher-priority settings or explicit XDG_CONFIG_DIRS skip the probe. Docs in pdm-compatibility.md describe these limits.

Large test additions cover precedence, string booleans, site/user/global layers, and macOS launcher/probe safety.

^{Reviewed by Cursor Bugbot for commit 3226a8f. Configure here.}

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

[burn-down agent] Labeled Ready for review at 4521c098e73b2c1d1c7662a683775b1d9733fa17.

• CI: 335/335 completed check runs green (6 skipped by path filters), no failures.
• Bugbot: reviewed 4521c09 — no issues found; 0 unresolved review threads.
• Reviewer focus: python_crawler.rs: PDM settings now follow PDM's layered config (env > project > user > site) with ensure_boolean-style coercion of string values like "false".
• Slack announcement: not sent (Slack send tool unavailable in this run); next run will retry.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Review updated for 3226a8f2a8c851be496cb2efc3f59af8d5198949: Ready to merge as-is from this review. Final-head CI is complete: 335 successful checks, 6 skipped; no failures or pending checks. Bugbot passed, there are no unresolved review threads, and the PR is mergeable.

• macOS now honors the XDG site layer and obtains the implicit site path from PDM's own Python runtime, including the different Homebrew default. The probe is bounded, uses a private working directory, strips Python bootstrap/import overrides and does not invoke PDM or its plugin loader.
• Legacy .pdm.toml values retain PDM's precedence over modern string values in pdm.toml, with missing-key fallback and PDM_USE_VENV priority preserved.

All 77 focused Python crawler tests and 18 native runtime controls passed. The native controls cover uv and spaced-path Homebrew PDM installations plus a HOME-based user-site platformdirs fixture under six inherited-environment scenarios; no planted code ran. Native PDM layer controls also passed with platformdirs 4.9.6 and 4.12.2. Formatting/diff checks and targeted clippy passed, with existing baseline formatting and macOS warning allowances documented in the PR.

Two independent reviews found no remaining actionable issue. The committed files match the tested sources and merge cleanly with current main. Unrecognized macOS launchers or a failed runtime probe leave the implicit site layer undiscovered; explicit XDG configuration still works, and this limitation is documented.

No remaining code finding from this review. The Ready label has been restored after all checks completed on the corrected commit.

[Mikola Lysenko (mikolalysenko)]

BugBot review

Please review the corrected commit 3226a8f2a8c851be496cb2efc3f59af8d5198949.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 3226a8f. Configure here.}