Add CI test for cross-builds

Feature or enhancement

Proposal:

Cross-builds depend on a fair bit of sysconfig internals, it would be good have a test in the CI to ensure they work.

This would have prevented #127178 (comment).

I don't think there's a need to run the full test suite, we should be fine running just the sysconfig, site, and test_embed tests. It would also make sense to run the tests in the build directory, and after installation.

The test should consist of:

Building a host Python, specifying a custom prefix (eg. workdir/host-python)
Installing the host Python
Configuring a new build using --with-build-python=workdir/host-python/bin/python, and also using a custom prefix (eg. workdir/cross-python)
Running the tests in the build directory (./python -m test test_sysconfig test_site test_embed)
Installing the cross-build
Running the tests again, but with the installed Python (workdir/cross-python/bin/python -m test test_sysconfig test_site test_embed)

I think we should be fine with Linux-only, but Windows would also be welcome. Additionally, possibly, we could also use a different architecture in the cross-build, like jit.yml does for "Emulated Linux".

cpython/.github/workflows/jit.yml

Lines 129 to 146 in 38264a0

    
                 - name: Emulated Linux 
        
                   if: runner.os == 'Linux' && matrix.architecture != 'x86_64' 
        
                   # The --ignorefile on ./python -m test is used to exclude tests known to fail when running on an emulated Linux. 
        
                   run: | 
        
                     sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh ${{ matrix.llvm }} 
        
                     export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH" 
        
                     ./configure --prefix="$(pwd)/../build" 
        
                     make install --jobs 4 
        
                     make clean --jobs 4 
        
                     export HOST=${{ matrix.architecture }}-linux-gnu 
        
                     sudo apt install --yes "gcc-$HOST" qemu-user 
        
                     export QEMU_LD_PREFIX="/usr/$HOST" 
        
                     CC="$HOST-gcc" \ 
        
                       CPP="$HOST-gcc --preprocess" \ 
        
                       HOSTRUNNER=qemu-${{ matrix.architecture }} \ 
        
                       ./configure --enable-experimental-jit ${{ matrix.debug && '--with-pydebug' || '' }} --build=x86_64-linux-gnu --host="$HOST" --with-build-python=../build/bin/python3 --with-pkg-config=no ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes 
        
                     make all --jobs 4 
        
                     ./python -m test --ignorefile=Tools/jit/ignore-tests-emulated-linux.txt --multiprocess 0 --timeout 4500 --verbose2 --verbose3

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

Add CI test for cross-builds

pythongh-127432: Add CI job to cross build Python

fixup! pythongh-127432: Add CI job to cross build Python

squash! pythongh-127432: Add CI job to cross build Python

fixup! pythongh-127432: Add CI job to cross build Python

pythongh-127432: Add CI job to cross build Python

gh-127432: Add CI job to cross build Python (#128380)

pythonGH-127432: add cross-build-linux as a required CI job

pythongh-127432: Add CI job to cross build Python (python#128380)

GH-127432: add cross-build-linux as a required CI job (#129459)

Triage: anything more to do here, or shall we close this issue?

Yes, this should be good, we can close 😄

Navigation Menu

Repository navigation

Add CI test for cross-builds #127432

Feature or enhancement

Proposal:

Has this already been discussed elsewhere?

Links to previous discussion of this feature:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	- name: Emulated Linux
	if: runner.os == 'Linux' && matrix.architecture != 'x86_64'
	# The --ignorefile on ./python -m test is used to exclude tests known to fail when running on an emulated Linux.
	run: \|
	sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh ${{ matrix.llvm }}
	export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
	./configure --prefix="$(pwd)/../build"
	make install --jobs 4
	make clean --jobs 4
	export HOST=${{ matrix.architecture }}-linux-gnu
	sudo apt install --yes "gcc-$HOST" qemu-user
	export QEMU_LD_PREFIX="/usr/$HOST"
	CC="$HOST-gcc" \
	CPP="$HOST-gcc --preprocess" \
	HOSTRUNNER=qemu-${{ matrix.architecture }} \
	./configure --enable-experimental-jit ${{ matrix.debug && '--with-pydebug' \|\| '' }} --build=x86_64-linux-gnu --host="$HOST" --with-build-python=../build/bin/python3 --with-pkg-config=no ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes
	make all --jobs 4
	./python -m test --ignorefile=Tools/jit/ignore-tests-emulated-linux.txt --multiprocess 0 --timeout 4500 --verbose2 --verbose3

Add CI test for cross-builds #127432

Description

Feature or enhancement

Proposal:

Has this already been discussed elsewhere?

Links to previous discussion of this feature:

Linked PRs

Activity

hugovk commented on Feb 13, 2025

FFY00 commented on Feb 13, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions