Repository navigation
How to link with a non-system libffi? #58732
Description
| BPO | 14527 |
|---|---|
| Nosy | @ned-deily, @pmp-p, @rupertnash, @munocat, @thesamesam |
| PRs |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2012-04-08.02:25:34.547>
labels = ['build', '3.9', '3.10', '3.11', 'ctypes', 'type-feature']
title = 'How to link with a non-system libffi?'
updated_at = <Date 2021-10-29.16:13:24.306>
user = 'https://bugs.python.org/pda'bugs.python.org fields:
activity = <Date 2021-10-29.16:13:24.306>
actor = 'thesamesam'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build', 'ctypes']
creation = <Date 2012-04-08.02:25:34.547>
creator = 'pda'
dependencies = []
files = []
hgrepos = []
issue_num = 14527
keywords = ['patch']
message_count = 13.0
messages = ['157776', '157777', '157795', '157932', '158004', '368399', '368413', '370088', '370580', '376885', '381577', '381601', '389390']
nosy_count = 10.0
nosy_names = ['ned.deily', 'pmpp', 'rosslagerwall', 'pda', 'rnash', 'Joshua Merchant', 'munocat', 'zhao.wang.unsw', 'raphael.kl', 'thesamesam']
pr_nums = ['20451']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue14527'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']
Activity
pda commentedon Apr 8, 2012
I trying to build python using an external libffi package I have installed -- is there some trick in directing --with-system-ffi to the path where it's located. I don't see clues in config.log or anywhere to help.
rosslagerwall commentedon Apr 8, 2012
If it is in a non-standard location, try setting the environment variables:
LDFLAGS linker flags, e.g. -L<lib dir> if you have libraries in a nonstandard directory <lib dir>
CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if you have headers in a nonstandard directory <include dir>
pda commentedon Apr 8, 2012
On Sun, Apr 08, 2012 at 05:58:29AM +0000, Ross Lagerwall wrote:
I'm pretty sure I've already been doing that, but I'll verify again soon
when I get the chance.
pda commentedon Apr 10, 2012
Yes, I think my libffi setup is okay, but python apparently doesn't (according to the deeper-down log files I didn't initially know about). The following is a suspicious-looking snippet from build/temp.hp-ux-B.11.31-ia64-2.7/libffi/config.log... I have to question the usefulness of that linker error message. My immediate thought is that maybe conftstm.o is a 32-bit object file, but I don't see anything earlier in the log to indicate it was even created.
configure:6159: gcc463 -o conftest -I. -IInclude -I./Include -D_TERMIOS_INCLUDED -I/usr/local/lp64/include -mlp64 -L/usr/local/src/Python-2.7.2 -L/usr/local/lp64/lib conftest.c conftstm.o >&5
ld: Mismatched Data ABI. Expected EF_IA_64_ABI64 but found None in file conftstm.o
Fatal error.
pda commentedon Apr 11, 2012
While this is no solution by any means, I think it'd be better for the scenario to be a fatal configure error. After all, if I say --with-system-ffi, it means I really, really want want to use my own libffi.
ned-deily commentedon May 8, 2020
As of current Python 3 releases, like 3.8, the python build no longer vendors a copy of libffi for most Unix systems and for those systems the --with-system-ffi configure option is ignored, i.e. configure and setup.py will always try to find an external libffi. On most systems, if you use are OK with using the system-supplied libffi, building and execution should usually "just work". But if you want to use another version of libffi, for example, one that you build from source, it can be very non-intuitive of how to successfully do that *especially* if there is already a system libffi installed. Depending on the system, you may need to set environment build variables like LDFLAGS, CFLAGS or PKG_CONFIG_PATH, and LD_LIBRARY_PATH for the Python build and/or you might need to override some of the libffi build variables to install in the proper locations for your system. It seems like, as has been suggested in various places, setup.py should at least try to use pkg-config to find libffi library locations as it already does to find libffi include files. But that might not be enough to get a working setup, i.e. you still might need to set LD_LIBRARY_PATH or the equivalent to find libffi. At the very least, there should be some documentation on how to build with a non-system libffi.
[-]How to link with an external libffi?[/-][+]How to link with a non-system libffi?[/+]ned-deily commentedon May 8, 2020
See also earlier discussions in bpo-31710, bpo-34823, bpo-40488, and bpo-40535, closed as duplicated of this issue.
rupertnash commentedon May 27, 2020
I have just struggled with building CPython with the _ctypes module. Fundamentally, the problem appears to be that configure uses pkgconfig to find the libffi include directory, while setup.py's detect_ctypes only uses the global list of library directories.
I have made an attempt at fixing this by having configure produce the directory containing libffi (
LIBFFI_LIBDIR) and setup.py use this. However I've hardly any experience with autotools, so I would be very happy to be corrected if this is no use at all.The PR is #20451
4 remaining items
pmp-p commentedon Nov 22, 2020
I think the actual PR20451 with its proper use of pkgconfig could close https://bugs.python.org/issue31710 ( fixing at once android and wasm/wasi compilation )
rupertnash commentedon Mar 23, 2021
Due to some major changes in setup.py around libffi, I've had to re-implement my patch - please see Github. I'd appreciate any comments so we can get this merged. Thanks!
dilyanpalauzov commentedon Apr 15, 2022
I downloaded Python-3.10.4.tar.xz and applied https://github.com/python/cpython/pull/20451.diff . I want to compile Python to run in a chrooted environment. I call
With this patch _ctypes is linked correctly.
MhmRhm commentedon Dec 30, 2023
After compiling libffi from source with the following commands:
I was able to build _ctypes.
The mentioned command installs the binaries at /usr/lib/ instead of /usr/lib64/
encukou commentedon Mar 25, 2024
The setup changed again;
setup.pyand--with-system-ffiis no more. The environment variables to set are listed in./configure --help:For example:
LIBFFI_CFLAGS="-I $HOME/include/" LIBFFI_LIBS="-L $HOME/lib64/ -lffi"encukou commentedon Apr 1, 2024
I'll close the issue. Please open a new one if these environment variables don't work.