Skip to content

document cross compilation #72728

Closed
Closed
document cross compilation#72728
Assignees
xdegaye
Labels
3.7 (EOL)end of lifebuildThe build process and cross-builddocsDocumentation in the Doc dirtype-featureA feature request or enhancement

Description

@xdegaye
xdegaye
mannequin
opened on Oct 27, 2016
BPO 28542
Nosy @vadmium, @moreati
Files
  • readme.patch
  • readme_2.patch
  • readme_3.patch
  • readme_4.patch
  • readme_5.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/xdegaye'
closed_at = None
created_at = <Date 2016-10-27.10:17:40.433>
labels = ['type-feature', '3.7', 'build', 'docs']
title = 'document cross compilation'
updated_at = <Date 2019-12-10.08:13:06.397>
user = 'https://github.com/xdegaye'

    bugs.python.org fields:

    activity = <Date 2019-12-10.08:13:06.397>
actor = 'xdegaye'
assignee = 'xdegaye'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Cross-Build']
creation = <Date 2016-10-27.10:17:40.433>
creator = 'xdegaye'
dependencies = []
files = ['45238', '45263', '45268', '45294', '45319']
hgrepos = []
issue_num = 28542
keywords = ['patch']
message_count = 12.0
messages = ['279532', '279648', '279668', '279669', '279695', '279751', '279752', '279800', '279845', '279942', '280098', '280121']
nosy_count = 2.0
nosy_names = ['martin.panter', 'Alex.Willmer']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue28542'
versions = ['Python 3.6', 'Python 3.7']

    Activity

    1. xdegaye commented on Oct 27, 2016

      @xdegaye
      xdegayemannequin
      on Oct 27, 2016
      MannequinAuthor

      Patch adding a section to the README.

    2. xdegaye
      added
      3.7 (EOL)end of life
      on Oct 27, 2016
    3. xdegaye
      self-assigned this
      on Oct 27, 2016
    4. xdegaye
      added
      type-featureA feature request or enhancement
      docsDocumentation in the Doc dir
      buildThe build process and cross-build
      on Oct 27, 2016

    5. vadmium commented on Oct 29, 2016

      @vadmium
      vadmium
      on Oct 29, 2016
      Member

      Thanks, this looks like a decent start. I’m not that familiar with cross-compiling Python, but here are a couple questions:

      $(cd /path/to/source && pwd)/configure
      Why do you use $(pwd) here? I have mainly seen relative paths used (my own experiments, and reports from others), so I don’t think you need to force an absolute path. The instructions further up already mention “../configure” from a subdirectory.

      --prefix=$(cd ../install && pwd)
      Won’t that embed the path of the install directory in e.g. sys.prefix or something? Can’t you use “DESTDIR=../install” instead?

      I remember there are a bunch of extra things you have to manually configure to cross-compile. See e.g. revision 12a56a349af2 which asks for /dev/ptmx and /dev/ptc settings in a CONFIG_SITE file (although I have had success just adding them to the “configure” command line). Perhaps we could add something like

      '''
      configure --host=[. . .] \
      ac_cv_file__dev_ptmx=no \
      ac_cv_file__dev_ptc=no
      . . .

      If the target Python could use /dev/ptmx or /dev/ptc to implement os.openpty(), set the corresponding argument to "yes".
      '''

      I suspect some people cross compile 2.7, so it may be worth applying something like this to that branch.

    6. xdegaye commented on Oct 29, 2016

      @xdegaye
      xdegayemannequin
      on Oct 29, 2016
      MannequinAuthor

      $(cd /path/to/source && pwd)/configure
      Why do you use $(pwd) here?

      'configure' fails with the following error message when used with '--prefix=../install':
      configure: error: expected an absolute directory name for --prefix: ../install
      So for consistency, the example used absolute paths everywhere.

      --prefix=$(cd ../install && pwd)
      Won’t that embed the path of the install directory in e.g. sys.prefix or something? Can’t you use “DESTDIR=../install” instead?

      You are right.
      On an Android device where /usr and /bin do not exist, the platform independent Python files are installed on the sdcard. So the build system I am using for Android does the following:

      • uses '--prefix=/sdcard/org.bitbucket.pyona
      • before running 'make install', it creates temporarily the /sdcard directory on the build system (with sudo) and use 'sudo mount --bind $INSTALL_ROOT /sdcard' so that 'make install' actually copies the files to $INSTALL_ROOT and so that the modules are byte compiled modules with the proper '-d' option to compileall. This allows also for multiple cross-compilation using different $INSTALL_ROOT directory names for each Android API level or architecture (x86, arm, armv7, ...)
      • creates a tar file from the files in $INSTALL_ROOT that is later copied and expanded on the device using the Android swiss army knife 'adb shell'.

      I remember there are a bunch of extra things you have to manually configure to cross-compile. See e.g. revision 12a56a349af2 which asks for /dev/ptmx and /dev/ptc settings in a CONFIG_SITE file (although I have had success just adding them to the “configure” command line). Perhaps we could add something like

      '''
      configure --host=[. . .] \
      ac_cv_file__dev_ptmx=no \
      ac_cv_file__dev_ptc=no
      . . .

      If the target Python could use /dev/ptmx or /dev/ptc to implement os.openpty(), set the corresponding argument to "yes".

      Ok

      I suspect some people cross compile 2.7, so it may be worth applying something like this to that branch.

      Not sure that all the recent cross-compilation changes have been applied to 2.7. Would not documenting cross-compilation in 2.7 entail that we support it on 2.7 ?

    7. xdegaye commented on Oct 29, 2016

      @xdegaye
      xdegayemannequin
      on Oct 29, 2016
      MannequinAuthor

      New patch.

    8. xdegaye commented on Oct 29, 2016

      @xdegaye
      xdegayemannequin
      on Oct 29, 2016
      MannequinAuthor

      I have changed my Android build system to use DESTDIR, it is simpler than using 'mount --bind'. Thanks for the suggestion Martin.

      Here is a new patch adding some text for the Android cross compilation.

    9. xdegaye commented on Oct 30, 2016

      @xdegaye
      xdegayemannequin
      on Oct 30, 2016
      MannequinAuthor

      One must also add '--(en|dis)able-ipv6' to configure, otherwise configure exits with:
      Fatal: You must get working getaddrinfo() function.
      or you can specify "--disable-ipv6".

      The ACTION-IF-CROSS-COMPILING parameter of the AC_RUN_IFELSE that checks for getaddrinfo is $ac_cv_buggy_getaddrinfo="no -- configured with --(en|dis)able-ipv6".

    10. xdegaye commented on Oct 30, 2016

      @xdegaye
      xdegayemannequin
      on Oct 30, 2016
      MannequinAuthor

      The ACTION-IF-CROSS-COMPILING parameter of the AC_RUN_IFELSE that checks for getaddrinfo is $ac_cv_buggy_getaddrinfo="no -- configured with --(en|dis)able-ipv6".

      Hum, I should have written:
      The ACTION-IF-CROSS-COMPILING parameter of the AC_RUN_IFELSE that checks for getaddrinfo is $ac_cv_buggy_getaddrinfo="no -- configured with --(en|dis)able-ipv6" when configure is run with --disable-ipv6 or --enable-ipv6.

    11. xdegaye commented on Oct 31, 2016

      @xdegaye
      xdegayemannequin
      on Oct 31, 2016
      MannequinAuthor

      New patch, less verbose and taking into account the previous posts.

    12. vadmium commented on Nov 1, 2016

      @vadmium
      vadmium
      on Nov 1, 2016
      Member

      Regarding 2.7, I guess it depends on your definition of “support”. People have been cross-compiling Python 2.7 longer than I have been working on Python. In the past I have tried to apply cross compilation fixes to 2.7. But if you are only confident about 3.6, let’s just stick with that for the moment.

      FWIW, DESTDIR, --prefix, etc are not specific to cross compilation or Android. DESTDIR is also useful when building a package/tarball or whatever, rather than installing directly. And --prefix would be useful if you don’t have root access and want to install to $HOME or somewhere. These settings are mentioned in the 2.7 README, but it looks like Guido trimmed them out for Python 3.0 (revision 07d67a9725a7).

    13. xdegaye commented on Nov 2, 2016

      @xdegaye
      xdegayemannequin
      on Nov 2, 2016
      MannequinAuthor

      About 2.7, this is a list of the issues I am aware of, that may have to be fixed also in 2.7 (and 3.5 for some of them): issues bpo-26884 bpo-27434 bpo-28444 and bpo-22724. Issue bpo-28589 was entered today and is a duplicate of the previous one.

      FWIW, DESTDIR, --prefix, etc are not specific to cross compilation or Android.

      I must be seeing everything through my little Android hammer these days :)
      This new patch removes the last paragraph.

    14. vadmium commented on Nov 5, 2016

      @vadmium
      vadmium
      on Nov 5, 2016
      Member

      Actually I think it is good to document DESTDIR (you weren’t seem aware of it before?). I was just pointing out that it is useful to more than just Android. Perhaps Guido shouldn’t have trimmed it from the 2.7 readme (original text: https://hg.python.org/cpython/diff/988b3e807043/README). On the other hand, --prefix etc are already mentioned in “./configure --help”, and more people already seem to know about it.

    15. xdegaye commented on Nov 5, 2016

      @xdegaye
      xdegayemannequin
      on Nov 5, 2016
      MannequinAuthor

      Yes, I was not aware of DESTDIR. The 2.7 README contains also useful information on the build process, for example about some use cases of the '*shared*' entries in the Setup files (for when setup.py cannot detect/build an extension module) or about the '*static' entries (to be able to profile extension modules). It also explains Setup.local whereas the only information on Setup.local in the Python3 documentation, is restricted to https://docs.python.org/3/extending/extending.html#compilation-and-linkage :(

      There is also a lot of noise in the 2.7 README, this possibly explains the trimming. Maybe the cross-build section in the 3.6 README could be a link to a new build section in the documentation ?

      --prefix etc are already mentioned in “./configure --help”

      The documentation on sys.prefix and sys.exec_prefix gives details about where go which files which is useful when trying to figure out where to copy the files on the target after a cross-build.

    16. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    17. methane commented on Sep 30, 2026

      @methane
      methane
      on Sep 30, 2026
      Member

      29835 added cross-compilation documentation, including configuration options and an example.

    18. methane
      closed this as completedon Sep 30, 2026
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.7 (EOL)end of lifebuildThe build process and cross-builddocsDocumentation in the Doc dirtype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions