Skip to content

New site.py implementation fails to load .pth files in subprocesses #149819

Closed
Closed
New site.py implementation fails to load .pth files in subprocesses#149819
Assignees
warsaw
Labels
stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Description

@scoder
scoder
opened on May 14, 2026

Bug report

Bug description:

After #149583, .pth files are no longer loaded in Python subprocesses, as started with subprocess.run([sys.executable, …]) and the like.

The new implementation deliberately ignores .pth files in sys.path directories that are already known, and Python subprocesses already know all the paths of their parent process. Thus, all .pth files in those paths are ignored by subprocesses now.

I specifically noticed this when running setuptools in a build subprocess, which now fails to install its distutils replacement hack.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

Activity

  1. scoder
    assigned
    warsaw
    on May 14, 2026
  2. scoder
    added
    type-bugAn unexpected behavior, bug, or error
    on May 14, 2026
  3. scoder
    changed the title [-]New site.py implementations fails fails to load .pth files in subprocesses[/-] [+]New site.py implementations fails to load .pth files in subprocesses[/+] on May 14, 2026
  4. StanFromIreland
    added
    stdlibStandard Library Python modules in the Lib/ directory
    on May 14, 2026

  5. warsaw commented on May 15, 2026

    @warsaw
    warsaw
    on May 15, 2026
    Member

    It works for me with the following test script.

    #!/bin/sh
#
# Usage:
#   ./149819.sh                  # uses ./python.exe
#   ./149819.sh /usr/bin/python3 # uses a specific interpreter

PYTHON=${1:-./python.exe}

VENV=$(mktemp -d)
"$PYTHON" -m venv "$VENV"
SITE=$("$VENV"/bin/python -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')
cat > "$SITE"/pp.pth <<EOF
import sys; print('pp.pth')
EOF

"$VENV/bin/python" -c "import sys, subprocess; print('parent'); subprocess.run([sys.executable, '-c', 'print(\'child\')'])"

    This outputs:

    pp.pth
parent
pp.pth
child

  6. warsaw commented on May 15, 2026

    @warsaw
    warsaw
    on May 15, 2026
    Member

    This is using the head of the 3.15 branch 21909e8

  7. IntentBug
    added a commit that references this issue on May 15, 2026

    pythongh-149819: Fix .pth files not loaded in Python subprocesses

    b7f6ba1

  8. warsaw commented on May 18, 2026

    @warsaw
    warsaw
    on May 18, 2026
    Member

    Closing as Can't Reproduce.

  9. warsaw
    closed this as completedon May 18, 2026

  10. scoder commented on May 19, 2026

    @scoder
    scoder
    on May 19, 2026
    ContributorAuthor

    Closing as Can't Reproduce.

    Don't expect everyone else to work 24/7 on fixing other people's code. :]

    Here's the easy (for me) reproducer:

    git clone https://github.com/cython/cython
cd cython
python3.16 -m venv venv
. venv/bin/activate
python3 -m pip install setuptools
python3 runtests.py --debug "compile\.declarations"

    Fails in a file that the test run generates with

    Traceback (most recent call last):
  File "…/TEST_TMP/compile/declarations/setup.py", line 2, in <module>
    from distutils.core import setup
ModuleNotFoundError: No module named 'distutils'

    I was using CPython's main branch at f1a47e7 for this (that's shortly after #149659 was merged).

  11. scoder
    reopened this on May 19, 2026

  12. warsaw commented on May 20, 2026

    @warsaw
    warsaw
    on May 20, 2026
    Member

    Could you please try again with 87a879f from main or ad2f0cb from the 3.15 branch? I tried your recipe in both and it still WFM. It probably doesn't matter, but I'm on macOS.

    % python3 runtests.py --debug "compile\.declarations"
Python 3.15.0b1+ (heads/3.15:ad2f0cb997a, May 20 2026, 10:11:23) [Clang 21.0.0 (clang-2100.1.1.101)]

Running tests against Cython 3.3.0a0 7eeaf92d5d0c5ee1c2964566856b4d0568e8532c
Using Cython language level 2.
Test dependency not found: 'numpy'
Test dependency not found: 'pythran'
Test dependency not found: 'setuptools.sandbox'
Test dependency found: 'asyncio' version 3.15.0b1+
Test dependency found: 'pstats' version 3.15.0b1+
Test dependency found: 'posix' version 3.15.0b1+
Test dependency found: 'array' version ?.?
Test dependency not found: 'Cython.Coverage'
Test dependency not found: 'Cython.Coverage'
Test dependency not found: 'IPython.testing.globalipapp'
Test dependency not found: 'jedi_BROKEN_AND_DISABLED'
Test dependency found: 'test.support' version 3.15.0b1+
Backends: c,cpp

[etoe.declarations: SLOWTEST 0.88 sec] ----------------------------------------------------------------------
Ran 1 test in 0.888s

OK
Most expensive pipeline stages: 'AnalyseDeclarationsTransform': 0.15 / 8 (0.019 / run, 54.2%), 'parse': 0.03 / 8 (0.004 / run, 12.4%), 'generate_pyx_code_stage': 0.03 / 1 (0.031 / run, 11.1%), 'AnalyseExpressionsTransform': 0.01 / 8 (0.001 / run, 3.7%), 'PostParse': 0.01 / 8 (0.001 / run, 1.9%), 'OptimizeBuiltinCalls': 0.00 / 8 (0.001 / run, 1.6%), 'RemoveUnreachableCode': 0.00 / 16 (0.000 / run, 1.4%), 'InterpretCompilerDirectives': 0.00 / 8 (0.000 / run, 1.0%), 'MarkClosureVisitor': 0.00 / 8 (0.000 / run, 0.9%), 'ControlFlowAnalysis': 0.00 / 8 (0.000 / run, 0.9%)
Times:
etoe-build  :     0.66 sec  (   1,  0.663 / run) - slowest: 'c:declarations(1)' (0.66s)
etoe-run    :     0.22 sec  (   1,  0.217 / run) - slowest: 'c:declarations(2)' (0.22s)
ALL DONE

  13. scoder commented on May 20, 2026

    @scoder
    scoder
    on May 20, 2026
    ContributorAuthor

    I tried with latest main (87a879f) and it fails for me, as before. This is on Ubuntu Linux. We don't have "latest main" integration tests running on anything else, so I can't compare right now. But your failure to reproduce it speaks for a platform specific issue. Possibly some spawn/fork/exec kind of difference?

  14. warsaw
    added a commit that references this issue on May 21, 2026

    Merge branch 'main' into pythongh-149819-fix-pth-subprocess

    9c53ee4

  15. warsaw commented on May 21, 2026

    @warsaw
    warsaw
    on May 21, 2026
    Member

    Please review the latest linked PR

  16. scoder commented on May 21, 2026

    @scoder
    scoder
    on May 21, 2026
    ContributorAuthor

    Tried it, works for me, resolves this issue.

  17. scoder
    changed the title [-]New site.py implementations fails to load .pth files in subprocesses[/-] [+]New site.py implementation fails to load .pth files in subprocesses[/+] on May 21, 2026
  18. warsaw
    added 3 commits that reference this issue on May 21, 2026

    Merge branch 'main' into pythongh-149819-fix-pth-subprocess

    2575e1d

    gh-149819: fix .pth and .start file processing in subprocess when inh…

    3c298e2

    [3.15] gh-149819: fix .pth and .start file processing in subprocess w…

    233cf75

  19. warsaw commented on May 21, 2026

    @warsaw
    warsaw
    on May 21, 2026
    Member

    Fixed in main and 3.15 (it should show up in the next beta). Thanks again!

  20. warsaw
    closed this as completedon May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions