Skip to content

Subinterpreters can load modules without subinterpreter support #104621

Closed
Closed
Subinterpreters can load modules without subinterpreter support#104621
Assignees
ericsnowcurrently
Labels
3.12only security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)topic-importlibtopic-subinterpreterstype-bugAn unexpected behavior, bug, or error

Description

@brandtbucher
brandtbucher
opened on May 18, 2023

The first time a subinterpreter attempts to import a module without proper subinterpreter support, an ImportError is raised (which is correct). However, subsequent imports of the same module succeed!

Using readline as an example, since it's currently single-phase init:

>>> from _xxsubinterpreters import create, run_string
>>> s = "import readline; print(readline)"
>>> interp = create()
>>> run_string(interp, s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_xxsubinterpreters.RunFailedError: <class 'ImportError'>: module readline does not support loading in subinterpreters
>>> run_string(interp, s)
<module 'readline'>

Linked PRs

Activity

  1. brandtbucher
    added
    type-bugAn unexpected behavior, bug, or error
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    topic-subinterpreters
    topic-importlib
    3.12only security fixes
    on May 18, 2023
  2. brandtbucher
    assigned
    ericsnowcurrently
    on May 18, 2023
  3. github-project-automation
    added this to Subinterpreterson May 18, 2023
  4. github-project-automation
    moved this to Todo in Subinterpreterson May 18, 2023

  5. corona10 commented on May 21, 2023

    @corona10
    corona10
    on May 21, 2023
    Member
    
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8)
  * frame #0: 0x00000001000d02b0 python.exe`Py_TYPE(ob=0x0000000000000000) at object.h:204:16 [opt]
    frame #1: 0x00000001000d0e88 python.exe`Py_IS_TYPE(ob=<unavailable>, type=0x00000001004311e8) at object.h:235:12 [opt]
    frame #2: 0x00000001000cf590 python.exe`PyObject_TypeCheck(ob=0x0000000000000000, type=<unavailable>) at object.h:366:12 [opt]
    frame #3: 0x00000001000d043c python.exe`PyModule_GetState(m=0x0000000000000000) at moduleobject.c:592:10 [opt]
    frame #4: 0x0000000100ab6684 readline.cpython-312d-darwin.so`on_startup_hook at readline.c:1020:17 [opt]
    frame #5: 0x00000001d4f4d100 libedit.3.dylib`rl_initialize + 884
    frame #6: 0x0000000100ab4d88 readline.cpython-312d-darwin.so`setup_readline(mod_state=0x000000010210b4c0) at readline.c:1208:9 [opt]
    frame #7: 0x0000000100ab4b6c readline.cpython-312d-darwin.so`PyInit_readline at readline.c:1503:9 [opt]
    frame #8: 0x000000010019f574 python.exe`_PyImport_LoadDynamicModuleWithSpec(spec=0x000000010210b7b0, fp=0x0000000000000000) at importdl.c:169:9 [opt]
    frame #9: 0x000000010019f21c python.exe`_imp_create_dynamic_impl(module=<unavailable>, spec=0x000000010210b7b0, file=<unavailable>) at import.c:3726:11 [opt]
    frame #10: 0x000000010019e588 python.exe`_imp_create_dynamic(module=<unavailable>, args=0x00000001020b26d8, nargs=1) at import.c.h:506:20 [opt]
    frame #11: 0x00000001000ce478 python.exe`cfunction_vectorcall_FASTCALL(func=0x0000000102019310, args=0x00000001020b26d8, nargsf=1, kwnames=<unavailable>) at methodobject.c:422:24 [opt]
    frame #12: 0x00000001000879b8 python.exe`_PyVectorcall_Call(tstate=0x00000001100be588, func=(python.exe`cfunction_vectorcall_FASTCALL at methodobject.c:411), callable=0x0000000102019310, tuple=0x00000001020b26c0, kwargs=0x00000001020a22d0) at call.c:271:16 [opt]
    frame #13: 0x0000000100087b08 python.exe`_PyObject_Call(tstate=0x00000001100be588, callable=0x0000000102019310, args=0x00000001020b26c0, kwargs=0x00000001020a22d0) at call.c:354:16 [opt]
    frame #14: 0x0000000100087bcc python.exe`PyObject_Call(callable=<unavailable>, args=<unavailable>, kwargs=<unavailable>) at call.c:379:12 [opt]
    frame #15: 0x0000000100169ab8 python.exe`_PyEval_EvalFrameDefault(tstate=<unavailable>, frame=0x0000000100a80370, throwflag=<unavailable>) at bytecodes.c:3191:26 [opt]
    frame #16: 0x000000010015c888 python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:87:16 [opt]
    frame #17: 0x000000010015c7a8 python.exe`_PyEval_Vector(tstate=0x00000001100be588, func=0x000000010201cc50, locals=0x0000000000000000, args=0x000000016fdfe310, argcount=2, kwnames=0x0000000000000000) at ceval.c:1610:12 [opt]
    frame #18: 0x0000000100087d0c python.exe`_PyFunction_Vectorcall(func=0x000000010201cc50, stack=0x000000016fdfe310, nargsf=<unavailable>, kwnames=0x0000000000000000) at call.c:0:45 [opt]
    frame #19: 0x0000000100086f0c python.exe`_PyObject_VectorcallTstate(tstate=0x00000001100be588, callable=0x000000010201cc50, args=0x000000016fdfe310, nargsf=2, kwnames=0x0000000000000000) at pycore_call.h:92:11 [opt]
    frame #20: 0x00000001000889ec python.exe`object_vacall(tstate=0x00000001100be588, base=0x0000000000000000, callable=0x000000010201cc50, vargs="") at call.c:850:14 [opt]
    frame #21: 0x0000000100088884 python.exe`PyObject_CallMethodObjArgs(obj=<unavailable>, name=0x000000010050d438) at call.c:911:24 [opt]
    frame #22: 0x000000010019c9c4 python.exe`import_find_and_load(tstate=0x00000001100be588, abs_name=0x0000000102027e70) at import.c:2715:11 [opt]
    frame #23: 0x000000010019c1fc python.exe`PyImport_ImportModuleLevelObject(name=0x0000000102027e70, globals=<unavailable>, locals=<unavailable>, fromlist=0x0000000100431718, level=0) at import.c:2798:15 [opt]
    frame #24: 0x000000010016c9fc python.exe`import_name(tstate=0x00000001100be588, frame=0x0000000100a80020, name=0x0000000102027e70, fromlist=0x0000000100431718, level=0x0000000100505648) at ceval.c:2385:15 [opt]
    frame #25: 0x00000001001653a8 python.exe`_PyEval_EvalFrameDefault(tstate=<unavailable>, frame=0x0000000100a80020, throwflag=<unavailable>) at bytecodes.c:2077:19 [opt]
    frame #26: 0x000000010015c888 python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:87:16 [opt]
    frame #27: 0x000000010015c7a8 python.exe`_PyEval_Vector(tstate=0x00000001100be588, func=0x00000001020896d0, locals=0x00000001020a1f10, args=0x0000000000000000, argcount=0, kwnames=0x0000000000000000) at ceval.c:1610:12 [opt]
    frame #28: 0x000000010015c694 python.exe`PyEval_EvalCode(co=0x00000001020a6440, globals=0x00000001020a1f10, locals=0x00000001020a1f10) at ceval.c:567:21 [opt]
    frame #29: 0x00000001001c3334 python.exe`run_eval_code_obj(tstate=0x00000001100be588, co=0x00000001020a6440, globals=0x00000001020a1f10, locals=0x00000001020a1f10) at pythonrun.c:1695:9 [opt]
    frame #30: 0x00000001001c14a4 python.exe`run_mod(mod=<unavailable>, filename=<unavailable>, globals=0x00000001020a1f10, locals=0x00000001020a1f10, flags=<unavailable>, arena=<unavailable>) at pythonrun.c:1716:19 [opt]
    frame #31: 0x00000001001c0c20 python.exe`PyRun_StringFlags(str="import readline; print(readline)", start=257, globals=0x00000001020a1f10, locals=0x00000001020a1f10, flags=0x0000000000000000) at pythonrun.c:1591:15 [opt]
    frame #32: 0x0000000100a92a8c _xxsubinterpreters.cpython-312d-darwin.so`_run_script(interp=0x0000000110060010, codestr="import readline; print(readline)", shared=0x0000000000000000, sharedexc=0x000000016fdfe940) at _xxsubinterpretersmodule.c:425:24 [opt]
    frame #33: 0x0000000100a92880 _xxsubinterpreters.cpython-312d-darwin.so`_run_script_in_interpreter(mod=0x0000000100e92270, interp=0x0000000110060010, codestr="import readline; print(readline)", shareables=<unavailable>) at _xxsubinterpretersmodule.c:483:18 [opt]
    frame #34: 0x0000000100a925cc _xxsubinterpreters.cpython-312d-darwin.so`interp_run_string(self=0x0000000100e92270, args=<unavailable>, kwds=<unavailable>) at _xxsubinterpretersmodule.c:714:9 [opt]
    frame #35: 0x00000001000ced6c python.exe`cfunction_call(func=0x0000000100e93050, args=0x0000000100e8b660, kwargs=0x0000000000000000) at methodobject.c:537:18 [opt]
    frame #36: 0x00000001000871d8 python.exe`_PyObject_MakeTpCall(tstate=0x0000000100575858, callable=0x0000000100e93050, args=0x0000000100a78078, nargs=<unavailable>, keywords=0x0000000000000000) at call.c:240:18 [opt]
    frame #37: 0x0000000100086f48 python.exe`_PyObject_VectorcallTstate(tstate=0x0000000100575858, callable=0x0000000100e93050, args=0x0000000100a78078, nargsf=9223372036854775810, kwnames=0x0000000000000000) at pycore_call.h:90:16 [opt]
    frame #38: 0x0000000100087a24 python.exe`PyObject_Vectorcall(callable=<unavailable>, args=<unavailable>, nargsf=<unavailable>, kwnames=<unavailable>) at call.c:325:12 [opt]
    frame #39: 0x00000001001673b8 python.exe`_PyEval_EvalFrameDefault(tstate=<unavailable>, frame=0x0000000100a78020, throwflag=<unavailable>) at bytecodes.c:2643:19 [opt]
    frame #40: 0x000000010015c888 python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:87:16 [opt]
    frame #41: 0x000000010015c7a8 python.exe`_PyEval_Vector(tstate=0x0000000100575858, func=0x0000000100e316d0, locals=0x0000000100e23830, args=0x0000000000000000, argcount=0, kwnames=0x0000000000000000) at ceval.c:1610:12 [opt]
    frame #42: 0x000000010015c694 python.exe`PyEval_EvalCode(co=0x0000000100dedfe0, globals=0x0000000100e23830, locals=0x0000000100e23830) at ceval.c:567:21 [opt]
    frame #43: 0x00000001001c3334 python.exe`run_eval_code_obj(tstate=0x0000000100575858, co=0x0000000100dedfe0, globals=0x0000000100e23830, locals=0x0000000100e23830) at pythonrun.c:1695:9 [opt]
    frame #44: 0x00000001001c14a4 python.exe`run_mod(mod=<unavailable>, filename=<unavailable>, globals=0x0000000100e23830, locals=0x0000000100e23830, flags=<unavailable>, arena=<unavailable>) at pythonrun.c:1716:19 [opt]
    frame #45: 0x00000001001c0a80 python.exe`pyrun_file(fp=0x00000001ef988d10, filename=0x0000000100e8e180, start=<unavailable>, globals=0x0000000100e23830, locals=0x0000000100e23830, closeit=1, flags=0x000000016fdfef78) at pythonrun.c:1616:15 [opt]
    frame #46: 0x00000001001c0030 python.exe`_PyRun_SimpleFileObject(fp=0x00000001ef988d10, filename=0x0000000100e8e180, closeit=1, flags=0x000000016fdfef78) at pythonrun.c:433:13 [opt]
    frame #47: 0x00000001001bfd20 python.exe`_PyRun_AnyFileObject(fp=0x00000001ef988d10, filename=0x0000000100e8e180, closeit=1, flags=0x000000016fdfef78) at pythonrun.c:78:15 [opt]
    frame #48: 0x00000001001e2000 python.exe`pymain_run_file_obj(program_name=0x0000000100e8e1f0, filename=0x0000000100e8e180, skip_source_first_line=0) at main.c:360:15 [opt]
    frame #49: 0x00000001001e1bd0 python.exe`pymain_run_file(config=0x0000000100557b60) at main.c:379:15 [opt]
    frame #50: 0x00000001001e1384 python.exe`pymain_run_python(exitcode=0x000000016fdff0dc) at main.c:610:21 [opt]
    frame #51: 0x00000001001e120c python.exe`Py_RunMain at main.c:689:5 [opt]
    frame #52: 0x00000001001e1484 python.exe`pymain_main(args=0x000000016fdff140) at main.c:719:12 [opt]
    frame #53: 0x00000001001e14c8 python.exe`Py_BytesMain(argc=<unavailable>, argv=<unavailable>) at main.c:743:12 [opt]
    frame #54: 0x0000000100002054 python.exe`main(argc=<unavailable>, argv=<unavailable>) at python.c:15:12 [opt]
    frame #55: 0x000000010087908c dyld`start + 520

    I got crashed while I executed the following script on my macOS.

    from _xxsubinterpreters import create, run_string
s = "import readline; print(readline)"
interp = create()

try:
    run_string(interp, s)
except Exception as e:
    pass
run_string(interp, s)

  6. ericsnowcurrently commented on Jul 24, 2023

    @ericsnowcurrently
    ericsnowcurrently
    on Jul 24, 2023
    Member

    FYI, it turns out we were not doing the check properly for builtin modules.

  7. ericsnowcurrently
    added a commit that references this issue on Jul 27, 2023

    gh-104621: Check for Incompatible Extensions in import_find_extension…

    75c974f
  8. miss-islington
    added a commit that references this issue on Jul 27, 2023

    pythongh-104621: Check for Incompatible Extensions in import_find_ext…

    8911820
  9. ericsnowcurrently
    added a commit that references this issue on Jul 27, 2023

    [3.12] gh-104621: Check for Incompatible Extensions in import_find_ex…

    abaf89d
  10. ericsnowcurrently
    closed this as completedon Jul 27, 2023
  11. github-project-automation
    moved this from Todo to Done in Subinterpreterson Jul 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.12only security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)topic-importlibtopic-subinterpreterstype-bugAn unexpected behavior, bug, or error

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions