Repository navigation
Stack overflow error from recursion with 3.14.0rc1 on MacOS #137573
Description
Crash report
What happened?
In the pylint / astroid test suite we have a regression test for a common recursion issue on our end. Though the code triggers a deep recursion, so far CPython was still able to resolve it. While getting ready for 3.14, I noticed that a specific test started to crash Python but only on MacOS. Isolated Github action CI runs seem to confirm this, Ubuntu, Windows are completely fine both with 3.13.6 and 3.14.0rc1. However MacOS only passes with 3.13.6 while failing 3.14.0rc1. Noticeable 3.14.0a6 seems to be the last version which reliably still succeeded.
The releases have been downloaded from python.org; used the actions/setup-python action for tests in CI.
To bisect the issue further, I compiled 3.14.0rc1 on my Mac. Strangely though the test now passes, on the same system where the binary from python.org failed previously.
Local tests done with
- MacOS 14 on M2 Pro
Python 3.14.0rc1 (v3.14.0rc1:48f88310044, Jul 22 2025, 13:06:29) [Clang 16.0.0 (clang-1600.0.26.6)]from https://www.python.org/downloads/release/python-3140rc1/ => test failsPython 3.14.0rc1 (tags/v3.14.0rc1:48f88310044, Aug 9 2025, 00:54:21) [Clang 16.0.6 ]compiled manually => test succeeds
CC=/opt/homebrew/opt/llvm@16/bin/clang ./configure --with-openssl=$(brew --prefix openssl@3) --enable-optimizations --with-lto
make -j4 -sRun the test case
git clone --depth 1 --branch v4.0.0b2 https://github.com/pylint-dev/astroid.git
cd astroid
python -m venv venv
source venv/bin/activate
pip install pytest==8.4.1 .
pytest tests/test_inference.py::test_recursion_on_inference_tipOutput with 3.14.0rc1 from python.org
(venv) MyMacBook-Pro:astroid $ pytest -s tests/test_inference.py::test_recursion_on_inference_tip
============================== test session starts ===================================
platform darwin -- Python 3.14.0rc1, pytest-8.4.1, pluggy-1.6.0
rootdir: /.../astroid
configfile: pyproject.toml
collected 1 item
tests/test_inference.py Fatal Python error: _Py_CheckRecursiveCall: Unrecoverable stack overflow (used 3953 kB)
Python runtime state: initialized
Current thread 0x00000001dc071ec0 (most recent call first):
File "/.../astroid/astroid/decorators.py", line 81 in inner
File "/.../astroid/astroid/nodes/node_classes.py", line 451 in _infer
...
File "/.../astroid/astroid/decorators.py", line 44 in wrapped
File "/.../astroid/astroid/decorators.py", line 81 in inner
Abort trap: 6Output with 3.14.0rc1 compiled locally
tests/test_inference.py::test_recursion_on_inference_tip PASSED--
I'd be grateful about any pointers to things I'm doing "wrong" in my local build that prevent me from reproducing and bisecting the issue.
CPython versions tested on:
3.14
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
Python 3.14.0rc1 (v3.14.0rc1:48f88310044, Jul 22 2025, 13:06:29) [Clang 16.0.0 (clang-1600.0.26.6)]
Linked PRs
- GH-137573: Check C stack depth before stack allocating JIT optimizer struct #137676
- GH-137573: Add test to check that the margin used for overflow protection is larger than the stack space used by the interpreter #137724
- GH-137573: mark _PyOptimizer_Optimize as no inline #137731
- [3.14] GH-137573: mark _PyOptimizer_Optimize as no inline (GH-137731) #137751
- [3.14] GH-137573: Add test to check that the margin used for overflow protection is larger than the stack space used by the interpreter (GH-137724) #139295
Metadata
Metadata
Assignees
Labels
Projects
- StatusShow more project fieldsDone
Activity
inventshah commentedon Aug 10, 2025
I think the key change between
3.14.0a6and3.14.0b1is compiling with--enable-experimental-jit=yes-off(#131751 (review)).Reproduce locally with:
./configure -C --enable-experimental-jit=yes-off --with-lto=full --enable-optimizations. Note all three flags seem necessary.I ran
git bisectbetween bfcbb28 (last commit that touchesTools/jit/_llvm.py) and 1878fe8. The first bad commit is: a5274cd.One maybe relevant tidbit the JIT docs mention LLVM 19 as required but the python.org MacOS builds are compiled with clang 16?
inventshah commentedon Aug 11, 2025
Looks like builtin dunder calls cause the recursion limit to be hit much sooner than it should. Removing
__getattribute__fromUninferableBasefixes the test in OP, but breaks other tests. In the other issue forpyclibrary, removing one__str__call in a dependency also fixes the test reported.A minimal repro is
This
abortsafter 144 calls, while redefiningstrlikedef str(x): return x.__str__(), reportsRecursionError: maximum recursion depth exceededas expected (and completes the 1000 allowed). 3.13 raises aRecursionErrorin both cases.@hugovk since you asked about a minimal code example in the other issue, and I think these are all related.
AA-Turner commentedon Aug 11, 2025
Confirming that on Windows the example fails properly with a
RecursionError:AA-Turner commentedon Aug 11, 2025
cc @markshannon for a5274cd (GH-135225)
inventshah commentedon Aug 11, 2025
I guess the problem isn't exactly in recursion, but rather something about the stack use of dunder calls. I had also set up a chain of 144 nested classes that subsequently call a child's str (something that should not error, and is probably more akin to whats happening in the failed tests). That hits
RecursionError: Stack overflow, while one fewer nested class works.Click to expand class generator
encukou commentedon Aug 11, 2025
#131543 looks similar.
inventshah commentedon Aug 11, 2025
Looks like in
3.14.0b1we can have a chain of376classes, while in the3.14.0rc1only144. FWIW, there is this comment about lto in configure so maybe--with-lto=fullis broken (when I compile withthinit seems to be fine...)cpython/configure
Lines 8410 to 8413 in 3626928
Also, the main testsuite has failures, like
test_isinstance.TestIsInstanceIsSubclass.test_infinite_cycle_in_bases, so this should be catchable via CI.inventshah commentedon Aug 11, 2025
Ran another
git bisectwith the infinite recursion repro, and ac3c439 is the first bad commit (starting from 3.14.0a05), which looks more relevant than the other commit. Note I still need--enable-experimental-jit=yes-off, which the alphas aren't compiled with, so I think there may be another issue further back or toolchain bug.23 remaining items
[3.14] pythonGH-137573: mark _PyOptimizer_Optimize as no inline (pyth…
GH-137573: Add test to check that the margin used for overflow protec…
cdce8p commentedon Sep 24, 2025
Is there anything left here? I believe all has been resolved and this issue could be closed.
AA-Turner commentedon Sep 24, 2025
The test was merged, but has not been backported. cc @encukou if he wants to do so.
pythonGH-137573: Add test to check that the margin used for overflow …
encukou commentedon Sep 24, 2025
Yes, this makes sense for 3.14.1. #139295 is the backport
[3.14] GH-137573: Add test to check that the margin used for overflow…
hugovk commentedon Oct 10, 2025
The backport has been merged.
(A downside of keeping issues open to track backports is merging the backport and forgetting the issue :)
Add CondaPkg.toml to pin Python < 3.14 for NonlinearSolveSciPy
Add CondaPkg.toml to pin Python < 3.14 for NonlinearSolveSciPy
00666: pythonGH-139653: Only raise an exception (or fatal error) when…
00666: pythonGH-139653: Only raise an exception (or fatal error) when…