Repository navigation
ImportError: _ssl.so: undefined symbol: _PyModule_Add #108525
Description
Bug report
Environment
$ uname
Linux
$ python --version{,}
Python 3.11.4 (tags/v3.11.4:d2340ef257, Aug 26 2023, 22:42:28) [GCC 12.2.0]Description
Importing the ssl shared library from a Python 3.11.5 build from a Python 3.11.4 binary produces an ImportError due to an undefined symbol. Isn't CPython's ABI supposed to be fully compatible between patch releases?
Example
FROM debian
RUN apt update\
&& apt install --yes build-essential git libssl-dev zlib1g-dev
RUN git clone --branch v3.11.4 https://github.com/python/cpython 3.11.4\
&& cd 3.11.4 && ./configure && make
RUN git clone --branch v3.11.5 https://github.com/python/cpython 3.11.5\
&& cd 3.11.5 && ./configure && make
RUN mv 3.11.5/build/lib.linux-x86_64-3.11/_ssl.cpython-311-x86_64-linux-gnu.so\
3.11.4/build/lib.linux-x86_64-3.11/_ssl.cpython-311-x86_64-linux-gnu.so
RUN 3.11.4/python -m sslError
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "//3.11.2/Lib/ssl.py", line 100, in <module>
import _ssl # if we can't import it, let the error propagate
^^^^^^^^^^^
ImportError: //3.11.2/build/lib.linux-x86_64-3.11/_ssl.cpython-311-x86_64-linux-gnu.so: undefined symbol: _PyModule_Add
Observations
It looks like fced79f is the breaking commit, as per git bisect results.
Activity
hansegucker commentedon Aug 27, 2023
I also observed the issue on my system.
CorvinM commentedon Aug 27, 2023
Also able to reproduce between 3.11 branch and 3.11.4, looks like #106863 broke the ABI
sunmy2019 commentedon Aug 28, 2023
@serhiy-storchaka as the commit author
serhiy-storchaka commentedon Aug 28, 2023
I don't think this warranty extends to the private API.
If you only use the public C API in your extension, you can expect forward- and backwards-compatibinility.
Clarify stable ABI documentation to exclude internal C libraries
Templarrr commentedon Aug 31, 2023
I see the same error locally in poetry after my Ubuntu upgraded 3.11.? -> 3.11.5. Any advice how this can be mitigated/fixed? It doesn't look like I can downgrade local python back to 3.11.4 - deadsnakes ppa don't list 3.11.4 anymore
LostInDarkMath commentedon Aug 31, 2023
@Templarrr I had the same issue and fixed it by updating poetry to the latest version.
erlend-aasland commentedon Aug 31, 2023
cc. RM @pablogsal
Templarrr commentedon Aug 31, 2023
@LostInDarkMath thank you! Seems to be the solution
encukou commentedon Aug 31, 2023
The reproducer looks contrived. In what situation would pieces of CPython get mixed up like this? Is Ubuntu's
_sslmodule not built together with the interpreter?CorvinM commentedon Aug 31, 2023
A faulty install or packaging script could easily cause this (but wouldn't be noticed in most cases, since they are usually ABI compatible) because the file paths are named identically (I.e., specific to 3.11, not 3.11.5).
CorvinM commentedon Aug 31, 2023
In this case it looks like a problem with poetry packaging (or its pythonpath?)
0x2b3bfa0 commentedon Aug 31, 2023
The original issue was also related to Python + Poetry, by the way.
0x2b3bfa0 commentedon Sep 1, 2023
This. ☝🏼 💯
zvolsky commentedon Sep 14, 2023
My
poetry --versionwas 1.6.1.poetry self updateexactly as other poetry commands has failed with this issue.I decided to "update" via
curl -sSL https://install.python-poetry.org | python3 -, i.e. reinstall,the
poetry --versionis now again 1.6.1.And Poetry works again.
(Debian Testing)
dusty-phillips commentedon Oct 6, 2023
I encountered this issue in a python3.11-slim Dockerfile even after updating poetry.
In my case, the issue was that the Dockerfile in question was installing the python3-dev apt package, which was providing 3.11.2 headers. I was able to remove this dependency without any repercussions.
Was devilishly hard to track down so thought I'd share in case it helps someone else.
Adapt omd update regarding issue with python 3.11.5
cove9988 commentedon Nov 14, 2023
Yes, the old poetry 1.6.1 is not working well with python 3.11
if you
curl -sSL https://install.python-poetry.org | python3 -upgrade to 1.7.1, It works well.vstinner commentedon Dec 2, 2023
Usually, a Python binary loads its "own" Python stdlib. I don't think that we want to support using a newer/older Python with newer/older stdlib. You should use the exact same version for the Python binary and its stdlib.
What can be done is to enhance error reporting. For example, but storing the version in stdlib C extensions and raise an error if there is version mismatch.
Today, you get an ImportError because of a missing symbol. But tomorrow, you might get a crash because of subtle different behavior :-(
vstinner commentedon Mar 18, 2024
I close the issue. I don't think that we want to support this use case.
Update base image to Python 3.13-bookworm