Repository navigation
Build changes for Windows free-threaded builds #112984
Description
We should use a different $(IntDir) and $(OutDir) for free threaded builds so that it's faster to switch between them and so they can be built in parallel.
The output filename should be python{major}.{minor}t.exe and python{major}{minor}t.dll. Debug builds are python{major}.{minor}t_d.exe and python{major}{minor}t_d.dll.
The ABI tag becomes .cp{major}{minor}t-{plat}
All stdlib .pyd files get an ABI tag.
The python.bat is updated as normal (overwriting the last yesgil build).
The installer should not include a python.exe, to avoid causing PATH conflicts, but Nuget, embeddable and Store packages should include a copy/alias.
PEP 514 registration (a.k.a. sys.winver) should use {major}.{minor}t-{plat}.
Tests that rely on the executable name should use sys.executable wherever possible, and hopefully there's a support variable already for the rest.
All of these only take effect when $(DisableGil) == 'true' (which is set by build.bat --disable-gil, but the build variable is canonical).
Activity
zooba commentedon Dec 14, 2023
Deliberately not updating
sys.version,sys.thread_infoor the opening prompt of the REPL in this PR, because those affect all platforms and other people will want to argue about them.pythongh-112984 Update Windows build for free-threaded builds
Merge remote-tracking branch 'upstream/main' into pythongh-112984
zooba commentedon Dec 20, 2023
So I've added an installer option in this PR
Normal first page:
Where you go after clicking "Customize":
The page with the new option (last checkbox):
It's labelled "Download" because the files will be on python.org and downloaded on demand, so if you don't have internet access you won't get them. The installer already has a
/layoutoption to pre-download all its optional components, so it's nothing new here.After install, there's a
python3.13t.exeandpythonw3.13t.exeinstalled alongside the usualpython.exe/pythonw.exe. Right now, they're already registered as normal, which meanspy.exe -3.13twill select it whilepy.exe -3.13gets the usual version. There's also a "Python 3.13 (64-bit, free-threaded)" shortcut added to the start menu.Unfortunately,
py.exe -3will select the free-threaded one, which I don't like but don't really have a great way to solve it - the easiest way is to make it so you need something likepy.exe -V:PythonFreeThreaded/3.13to get the free-threaded build, which has the advantage thatpy.exe -V:PythonFreeThreaded/will get the latest free-threaded build available, but a disadvantage is that shebang parsing won't be able to select it (without writing special case code in the launcher, which I'd rather not do if it can be avoided). On balance, I think it's probably best to change the company name toPythonFreeThreadedor something like that - open to suggestions!Right now, venv will create
python3.13t.exein the new environment. I'm tempted to say it should createpython.exeinstead, as that will make it easier to use. It probably requires more significant changes to venv though, so I'd rather do it later. Differentvenvlauncherimplementations are needed because they "know" the name of the real executable will be, so we already do a sneaky rename. (Possibly that's being done wrong in the current PR...)I haven't yet created a Nuget package or embeddable distro for the files. I'm leaning toward
python-freethreadedandpythonarm64-freethreadedpackages on Nuget. This will really help out people running on CI, as they'll be able to easily get a set of binaries independent from what the CI image contains and without having to run an installer as part of their job. We could also just put a zip file of the binaries on python.org, but I fear it would clutter the list of downloads (which is already considered cluttered). No separate embeddable distro at this stage, people can build it easily enough if they want it.zooba commentedon Dec 20, 2023
Today's updates:
vsajip commentedon Dec 21, 2023
There's quite a lot of venv-related change - is there any discussion or design documentation anywhere? For example, about the
venvlauncherexecutable - what is the need/rationale for it, as it appears to be new in this PR?zooba commentedon Dec 21, 2023
The
venvlauncher.cis justlauncher.cboiled down to just the venv parts, and none of the py.exe parts. The only change is that the executable embeds the expected Python executable name at compile time, as we can't just assume it'll bepython[_d].exeanymore.The changes to
venv\__init__.pyshould just streamline things for Windows. We need to more carefully pick the right redirector, and we can't rely on the name beingpython.exeanymore, and trying to handle the symlink->copy fallback per-file was already pretty rough as we aren't copying the regular executables.The design is the same as it was before, it's mainly just refactoring how we get there.
Merge remote-tracking branch 'upstream/main' into pythongh-112984
encukou commentedon Jan 4, 2024
Do you plan to provide freethreaded debug (
td) binaries?That does start a combinatorial explosion of builds, but I figure that people will need all the debugger help they can get. (And they'll want to run benchmarks so building only
dtisn't appropriate.)zooba commentedon Jan 4, 2024
They're included with the installer is the current PR. No option, you just get a
python3.13t_d.exealong with thepython3.13t.exe. If size becomes an issue then they'll be the first thing to go, but I think they only add about 5MB to the installer package (which is only downloaded when the option is selected during install).I'm not convinced the debug binaries are that helpful anyway, compared to just building it yourself. If you're in a position where they are helpful, you almost certainly have the build tools already (or know that you're doing something weird and ought to have the build tools).
Merge remote-tracking branch 'upstream/main' into pythongh-112984
5 remaining items
pythongh-112984 Update Windows build and installer for free-threaded …
pythongh-112984: Fix test_ctypes.test_loading.test_load_dll_with_flag…
zooba commentedon Jan 22, 2024
@itamaro Yes, nogil tests now need to pass
--disable-giltort.batin order to get the correct executable name. I'm not sure where best to put that, I don't maintain any buildbots.itamaro commentedon Jan 22, 2024
the builder is configured here. is the solution to add
testFlags = ["--disable-gil"]? (I'm guessing these flags are passed down tort.bat?)itamaro commentedon Jan 22, 2024
we'll see if python/buildmaster-config#458 fixes it
pythongh-112984: Fix link error on free-threaded Windows build
gh-112984: Fix link error on free-threaded Windows build (GH-114455)
zooba commentedon Feb 1, 2024
I don't see any need to keep this open, so closing. Feel free to reopen if there's a reason.
pythongh-112984 Update Windows build and installer for free-threaded …
pythongh-112984: Fix test_ctypes.test_loading.test_load_dll_with_flag…
pythongh-112984: Fix link error on free-threaded Windows build (pytho…
pythongh-112984 Update Windows build and installer for free-threaded …
pythongh-112984: Fix test_ctypes.test_loading.test_load_dll_with_flag…
pythongh-112984: Fix link error on free-threaded Windows build (pytho…