Repository navigation
sys.path[0] Is Set Differently From the Rest of sys.path #109853
Description
Activity
We should consider calculating sys.path[0] and setting it to its own PyConfig field via getpath.py
There are two cases:
- (A)
PyConfig.run_filenameis set:pymain_get_importer()computessys.path[0] - (B) Otherwise,
_PyPathConfig_ComputeSysPath0()computessys.path[0]
For the case (A), do you want to execute PyImport_GetImporter() twice? Once in Py_InitializeFromConfig(), then again in pymain_run_python()? It's to decide if pymain_run_module() or pymain_run_file() should be used.
For case (B), this code path can be easily moved to Py_InitializeFromConfig(). When I designed and implemented PyConfig, I tried to minimize changes. But apparently, now the dust has settled, and we can go further :-)
- (A)
I think we should move most of the default
sys.pathcalculation intopython.c, including the running ofgetpath.py(we'd need to expose the ability to create and then close a runtime that can't import anything).If we're able to fully initialise the search path using only our public APIs, we'll have a much better interface for embedders to use.
I would love that sys.path would be fully initialized before the site module is loaded. Currently, sys.path is still modified by the site module in many ways, and so
python -Sgives a differentsys.path:-(site changes:
- Make paths absolute (why not doing that earlier?)
- Add user site directory (is it complicated to move the logic to getpath?)
I expect most of the site module can move into getpath. Venv and pth sure can (though we'd have to defer code execution in pth files until after initialization finishes). Some of the interactive mode features probably can't, but I'd also like to treat those as something specific to python.c and separate from libpython (i.e. part of the Python program not the Python interpreter).
This adds
PyConfig.sys_path_0as public API.
Should we add some documentation for it, or mark it internal (add an underscore)?Need @ericsnowcurrently to confirm, but I suspect marking it internal is better. When the calculation gets refactored into
getpath.pythen there shouldn't be any need to store it separately.This adds
PyConfig.sys_path_0as public API. Should we add some documentation for it, or mark it internal (add an underscore)?We should mark it as internal at least for now. We'd need to sort out the complexity I described above before this would become meaningful config.
The
sys.path[0]initialisation semantics are even worse than @ericsnowcurrently describes, sincerunpymay mutate the value if it gets invoked via-mor path entry execution.There's an intrinsic problem here in that
sys.path[0]is not semantically identical to othersys.pathentries (it can be set from a much wider variety of sources, including being dropped entirely when running in isolated mode), but once the desired value is figured out, we do want it to be treated the same as any other entry for module import purposes (hence it being in the list rather than stored somewhere else).I wonder if we can make
from . import <mod>work from__main__easily so there's a way to transition towards-P(nosys.path[0]by default)? Or if that's even worth attempting?
Feature or enhancement
Currently
sys.path[0]is set bypymain_run_python()(in Modules/main.c). This happens afterpymain_init(), which initializes the runtime, including the rest ofsys.path(via getpath.py and site.py). This makes it harder to reason about and introduces extra complexity for subinterpreters. (See gh-109793 and gh-109794.)We should consider calculating
sys.path[0]and setting it to its ownPyConfigfield via getpath.py, when the rest of the basesys.pathis calculated. We may need a later check to verify that there is a matching importer, aspymain_run_python()does. (FWIW, it isn't clear that there's any value to storing thesys.path[0]value on the global_PyPathConfig.)Also, we currently wait to actually set
sys.path[0](for the main interpreter) until after the readline/rlcompleter modules are imported inpymain_run_python(). We'd need to factor that in.CC @zooba, @vstinner, @ncoghlan
Linked PRs