Activity

1. pmitros

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Nov 11, 2022
2. AlexWaygood

   changed the title [-]Bug: mp.util.debug is called, but mp.util is never imported in process.py[/-] [+]concurrent/futures/process.py: mp.util.debug is called, but mp.util is never imported[/+] on Nov 11, 2022
3. gpshead commented on Nov 13, 2022

   

   gpshead

   on Nov 13, 2022

   Member

   More actions

   Was this perhaps at process shutdown called from a destructor?

   Notice that the error message is 'NoneType' object has no attribute 'util' which means that "mp" was None. So this isn't about mp.util not being imported, it's about "mp" in the context of when this ran being None. Which is possible in GC destructor context at process shutdown where a lot of the world has already been torn down, the kind of place that weakref callbacks may happen in.

   If you can reproduce this, the proper fix is not neccessarily to remove the call, but add mp_util=mp.util as a keyword argument to that weakref_cb function and use that instead of looking up mp.util from the global scope in the function body.
4. ktbyers commented on Dec 6, 2022

   

   ktbyers

   on Dec 6, 2022

   More actions

   This same issue started happening in a Python 3.9 environment of mine:

   Exception ignored in: <function _ExecutorManagerThread.__init__.<locals>.weakref_cb at 0x7f2926625e50>
   Traceback (most recent call last):
     File "/usr/local/lib/python3.9/concurrent/futures/process.py", line 285, in weakref_cb
   AttributeError: 'NoneType' object has no attribute 'util'
   

   My script looked like it was running properly and the issue was happening when the script was ending.

   I could fix the issue by doing the following:

           def weakref_cb(_,
                          thread_wakeup=self.thread_wakeup,
                          shutdown_lock=self.shutdown_lock):
               # I added this 'if mp:' check
               if mp: 
                   mp.util.debug('Executor collected: triggering callback for'
                                 ' QueueManager wakeup')
               with shutdown_lock:
                   thread_wakeup.wakeup()
5. ktbyers commented on Dec 6, 2022

   

   ktbyers

   on Dec 6, 2022

   More actions

   This fix mentioned above also worked for me:

           def weakref_cb(_,
                          thread_wakeup=self.thread_wakeup,
                          shutdown_lock=self.shutdown_lock,
                          mp_util_debug=mp.util.debug):
               mp_util_debug('Executor collected: triggering callback for'
                                 ' QueueManager wakeup')
               with shutdown_lock:
                   thread_wakeup.wakeup()

   @pmitros @gpshead
6. iritkatriel

   added

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   on Nov 26, 2023
7. encukou commented on Mar 19, 2024

   

   encukou

   on Mar 19, 2024

   Member

   More actions

   @ktbyers, do you want to send a PR?
8. vasiliyk commented on Sep 24, 2026

   

   vasiliyk

   on Sep 24, 2026

   Contributor

   More actions

   Fixed by gh-118112 (3.13+), which binds mp.util.debug as a default argument of weakref_cb as suggested above.

   Quick local test on main on MacOS:

   >>> import concurrent.futures.process as p
   ... 
   ... ex = p.ProcessPoolExecutor()
   ... ex.submit(pow, 2, 3).result()
   ... cb = ex._executor_manager_thread.executor_reference.__callback__
   ... p.mp, mp = None, p.mp
   ... cb(None) # No AttributeError
   ... p.mp = mp
   ... ex.shutdown()
   ... 
   >>> 

   I think this can be closed.
9. gpshead

   closed this as completedon Sep 25, 2026