Repository navigation
setup.py: do not add system header locations when cross compiling #64410
Description
| BPO | 20211 |
|---|---|
| Nosy | @doko42, @vstinner, @zware, @tpetazzoni, @yan12125 |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2014-01-09.23:02:08.889>
labels = ['build', '3.7']
title = 'setup.py: do not add system header locations when cross compiling'
updated_at = <Date 2019-12-10.08:12:27.238>
user = 'https://github.com/tpetazzoni'bugs.python.org fields:
activity = <Date 2019-12-10.08:12:27.238>
actor = 'xdegaye'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Cross-Build']
creation = <Date 2014-01-09.23:02:08.889>
creator = 'thomas-petazzoni'
dependencies = []
files = ['33393', '45830', '45893', '45894']
hgrepos = []
issue_num = 20211
keywords = ['patch']
message_count = 12.0
messages = ['207805', '237380', '261435', '282640', '282641', '282836', '282837', '282841', '282843', '283182', '283183', '283184']
nosy_count = 7.0
nosy_names = ['doko', 'vstinner', 'python-dev', 'zach.ware', 'thomas-petazzoni', 'yan12125', 'hundeboll']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue20211'
versions = ['Python 3.6', 'Python 3.7']
Activity
tpetazzoni commentedon Jan 9, 2014
In the cross-compilation case, setup.py incorrectly adds /usr/include to self.compiler.include_dirs, and results in the following invalid compilation line:
/home/thomas/projets/buildroot/output/host/usr/bin/arm-none-linux-gnueabi-gcc
-fPIC -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g
-O3 -Wall -Wstrict-prototypes -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-D_FILE_OFFSET_BITS=64 -pipe -Os
-I./Include -I/usr/include -I. -IInclude
-I/home/thomas/projets/buildroot/output/host/usr/arm-buildroot-linux-gnueabi/sysroot/usr/include
-I/home/thomas/projets/buildroot/output/build/python3-3.4.0b1/Include
-I/home/thomas/projets/buildroot/output/build/python3-3.4.0b1
-c /home/thomas/projets/buildroot/output/build/python3-3.4.0b1/Modules/_struct.c
-o build/temp.linux-arm-3.4/home/thomas/projets/buildroot/output/build/python3-3.4.0b1/Modules/_struct.o
cc1: warning: include location "/usr/include" is unsafe for cross-compilation [-Wpoison-system-directories]
The -I/usr/include is wrong when cross compiling, so we disable adding INCLUDEDIR and LIBDIR from the host when cross compiling.
BreamoreBoy commentedon Mar 6, 2015
Can we have a patch review on this issue please.
hundeboll commentedon Mar 9, 2016
The patch looks good to me, and works in my setup.
yan12125 commentedon Dec 7, 2016
Got the same issue here on building Python for Android. See https://circleci.com/gh/yan12125/python3-android/11 for an example build log. /usr/include is erroneously included and leads to build failures:
The patch looks reasonable and still applies to the default branch. With this patch the build is fine: https://circleci.com/gh/yan12125/python3-android/12
yan12125 commentedon Dec 7, 2016
Modified this issue a bit and add some experts on the build system
[-]setup.py: do not add invalid header locations[/-][+]setup.py: do not add system header locations when cross compiling[/+]xdegaye commentedon Dec 10, 2016
When cross compiling, setup.py must not look for headers in the directory for installing C header files and must not look for libraries in the directory for installing object code libraries of the _native_ interpreter which is being used to run setup.py.
Thomas patch updated to the tip of the default branch.
doko42 commentedon Dec 10, 2016
this assumption is wrong for the multiarch case.
yan12125 commentedon Dec 10, 2016
xdegaye's explanation is incorrect. The actual scene is much more complicated. First, let me explain why things may be broken. The wrong directory is included only if all of the following conditions are met:
os.path.normpath(sys.base_prefix) != '/usr'For example, if I use --prefix=/usr and DESTDIR=/tmp/python3-android/build, INCLUDEDIR will be /usr/include, which should not be included. On the other hand, if I use --prefix=/tmp/python3-android/build/usr and omit DESTDIR, INCLUDEDIR is correct /tmp/python3-android/build/usr/include. With $DESTDIR specified, the build script has the responsibility to specify $DESTDIR$PREFIX/include in CPPFLAGS, or CPython interpreter core won't even build, so skipping adding INCLUDEDIR in setup.py is safe. However, for the latter case (not using DESTDIR), the build script still needs to specify correct CPPFLAGS. I see it as a CPython limitation and worth documentation.
For Debian's multiarch framework, paths are already handled by add_multiarch_paths(). I'm not sure whether $PREFIX/include should still be included or not. (I don't use Debian-based systems)
xdegaye commentedon Dec 10, 2016
Please explain why it is wrong.
python-dev commentedon Dec 14, 2016
New changeset d1b400943483 by Xavier de Gaye in branch '3.6':
Issue bpo-20211: Do not add the directory for installing C header files and
https://hg.python.org/cpython/rev/d1b400943483
New changeset fcc9f19fcc13 by Xavier de Gaye in branch 'default':
Issue bpo-20211: Merge 3.6.
https://hg.python.org/cpython/rev/fcc9f19fcc13
xdegaye commentedon Dec 14, 2016
Sadly no explanations have been given, we have to take Matthias word for it.
This new patch updates the add_multiarch_paths() method and includes LIBDIR and INCLUDEDIR in their corresponding search paths for the multiarch case.
xdegaye commentedon Dec 14, 2016
Same patch, but this one is properly set to be handled by Rietveld.
serhiy-storchaka commentedon Dec 26, 2023
setup.pyno longer used to build CPython since 3.12 (see #94474).