Repository navigation
sysconfig data is generated with the host Python's Makefile when cross-compiling #127429
Description
Bug report
Bug description:
Currently, the POSIX sysconfig data is generated with the Makefile from the host interpreter, instead of the target. This results in incorrect data, as demonstrated by https://github.com/python/cpython/actions/runs/12090193539/job/33716932397?pr=127426.
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response
Activity
pythonGH-127429: fix sysconfig data generation on cross-builds
GH-127429: fix sysconfig data generation on cross-builds (#127430)
pythonGH-127429: fix sysconfig data generation on cross-builds (pytho…
zooba commentedon Dec 2, 2024
I still see this in my tests where I deliberately relocate Python after building.
Can
prefixbe made variable in the JSON file? I guess that means doing some reverse substitutions when emitting it.Alternatively, can we skip the test when the current
prefixdoesn't match the one in the JSON file?FFY00 commentedon Dec 2, 2024
We could ignore the prefix if it doesn't match the running interpreter.
Though, in your use-case, is the install expected to be relocated during its lifetime, or do you just do it as an intermediate step?
zooba commentedon Dec 2, 2024
I'm building it and packaging it up so that other people can download a pre-built interpreter into their builds. I can't control where they extract it to, so finding the prefix at runtime is basically necessary.
After it's been run for the first time it's unlikely to move again, but I don't get to have an executable install step. So ideally we'd just calculate the prefix each time we launch and never hard code it.
FFY00 commentedon Dec 2, 2024
Alright, I am not too worried about those use-cases. My recommendation would be to probably manually set the prefixes to
nullafter building.For all practical purposes, this file is not critical, we don't load data from it currently, and if we move towards that, we will be definitely keep calculating the prefix at runtime. Right now it is provided as a nice-to-have for introspection, but not for folks to rely on always existing.
Regarding the test, yep, we should definitely either ignore the prefixes when comparing the data, or skip it, so that it passes on the situation you described.
zooba commentedon Dec 2, 2024
Can you be more specific about what you mean by this? Do I try and patch the generated JSON file?
I wonder if there's a way we can override
configure --prefix=...to not generate a prefix at all? It won't install, obviously, but we should be able to build, right? (Or it will require a parameter tomake install, but still won't embed the prefix in the build.)FFY00 commentedon Dec 2, 2024
Yep. Either that, or just don't ship the JSON file. These are just my suggestions anyway, feel free to go for what you feel most comfortable with.
I think you should actually be able to build with an empty prefix (empty string), and install to a relocatable location using
DESTDIR.zooba commentedon Dec 2, 2024
Well this isn't an error I expected to see:
But yeah, passing an empty
--prefixgot way further than I expected. Might only be a few tweaks needed for that to be a viable approach.FFY00 commentedon Dec 2, 2024
Huh, interesting!
FFY00 commentedon Dec 12, 2024
Fixed via GH-127430. The relocating Python after build use-case should probably be covered in GH-119696.
pythonGH-127429: fix sysconfig data generation on cross-builds (pytho…
pythonGH-127429: fix sysconfig data generation on cross-builds (pytho…
[3.13] pythonGH-127429: fix sysconfig data generation on cross-builds (…