Repository navigation
Improve the docs regarding the migration from imp to importlib #104212
Description
Activity
Maybe we need a porting guide in the importlib docs for porting older imp and other APIs to importlib? I don't like that
impmodule has been deleted but the docs are still there.Reacted by Hugo van KemenadeIn current 3.12, 'imp' is no longer present in the module index, which suggests that it is gone.
I don't know the importlib design, so maybe it doesn't make sense, but... Would it make sense to add a imp_load_source() function to importlib? Maybe under a different name.
My own use case of load_module(): old code written first for Python 2 using imp then ported to SourceLoader. It's a short script which looks for "test_xxx.py" files and then run them. Stupid but simple pytest-like, good enough for my needs: https://github.com/vstinner/hachoir/blob/0a030e3c8045441a0c30a04475a94f3aadd818ed/runtests.py#L72
I was already annoyed to have to port to them a first date to get rid of imp. So the code should be updated again?
Would it make sense to add a imp_load_source() function to importlib?
Just to note, see, e.g., #58756 for some previous discussion of this.
- added a commit that references this issue
on May 24, 2023 The recipe didn't work for me:
- (1) The module has no
__file__variable - (2) The module was no registered in
sys.modules
imp.load_source()defines__file__and adds the module tosys.modules.I used this recipe instead:
def load_module(module_name, filename): loader = importlib.machinery.SourceFileLoader(module_name, filename) module = types.ModuleType(loader.name) module.__file__ = filename sys.modules[module.__name__] = module loader.exec_module(module) return module
Reacted by Alex Prengère, romainsc and mroelandts- (1) The module has no
- added a commit that references this issue
on Jun 13, 2023 - some solutions point to
importlib.util.spec_from_file_location, but this does not work with files that do not end with ".py"
That's actually only true if you don't pass in the loader you need to use as the code has to guess as what you're after. You can use either
importlib.util.spec_from_loader()orimportlib.util.spec_from_file_location(), they just need different specifics to do the right thing.Reacted by Alex Prengère- some solutions point to
- added a commit that references this issue
on Jun 16, 2023 20 remaining items
I'm not sure neither how find_module() was used with load_module() to load modules. Is the use case to specify a search path which is not in
sys.path?I think so.
Why not putting the path in
sys.pathand simply useimportlib.import_module()?Worried they would forget to clean up
sys.pathafter importing the one file they wanted?add it back to the docs, with explanations on how to migrate it to importlib
just document the migration in the release notes, in the section "Porting to Python 3.12"That's now done in What's New in Python 3.12, but under the "Removed" section, where removals are mentioned. I dislike documenting the same removal at two places.
https://docs.python.org/dev/whatsnew/3.12.html#removed
I just wrote a 3.12 backport for my second doc change: PR #106083. It'ss going to be merged soon.
I added a recipe to replace imp.load_source(). I consider that this documentation issue is now solved and I close the issue.
If someone wants a more complete explanation for a specific removed imp function, please open a new issue.
Thanks @alexprengere for the bug report, thanks @brettcannon and @arhadthedev for the reviews.
- added a commit that references this issue
on Mar 6, 2024 - added 7 commits that reference this issue
on Apr 28, 2024 The docs now state to replace
imp.load_source()with this:import importlib.util import importlib.machinery def load_source(modname, filename): loader = importlib.machinery.SourceFileLoader(modname, filename) spec = importlib.util.spec_from_file_location(modname, filename, loader=loader) module = importlib.util.module_from_spec(spec) # The module is always executed and not cached in sys.modules. # Uncomment the following line to cache the module. # sys.modules[module.__name__] = module loader.exec_module(module) return module
Is there a reason why this function was not directly put into
importlib.util?Is there a reason why this function was not directly put into
importlib.util?- There is a lot of variance people want for similar functionality as the code comment hints at.
- It avoids the import system while stuff in
importlib.utilis meant to help use it in some advanced ways. - Not everything needs to be in the stdlib.
Now that the
impremoval has landed on main, users migrating to Python3.12 will likely need help to move toimportlib.The current imp docs have tips on how to do just that. Great!
One caveat: the
imp.load_sourcehas been removed from the docs a long time ago, now it is only visible is the Python2 version of the docs. So users ofimp.load_sourcecannot rely on the docs to help them migrate, and have to Google this. The first results on stackoverflow are a bit wrong:SourceFileLoader(...).load_module(), but this is also deprecated and slated for removal in 3.12 (according to the warning)importlib.util.spec_from_file_location, but this does not work with files that do not end with ".py"The solution that I think is the best translation:
I think it would be beneficial to have that kind of information in the docs. Unfortunately,
imp.load_sourceis not officially documented, but there are several GitHub issues and SO threads discussing how to migrate code toimportlib. IMO, we should do one of:importlibWhat do you think?
Linked PRs