Skip to content

multiprocessing tests log: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown #89372

Closed
Closed
multiprocessing tests log: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown#89372
Labels
3.10 (EOL)end of life3.11only security fixes3.9 (EOL)end of lifetestsTests in the Lib/test dir

Description

@vstinner
vstinner
opened on Sep 15, 2021
BPO 45209
Nosy @vstinner, @ambv, @miss-islington, @sobolevn
PRs
  • bpo-45209: fix UserWarning: resource_tracker in test_multiprocessing #28377
  • [3.10] bpo-45209: fix UserWarning: resource_tracker in test_multiprocessing (GH-28377) #28499
  • [3.9] bpo-45209: fix UserWarning: resource_tracker in test_multiprocessing (GH-28377) #28500
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2021-09-21.18:37:39.102>
created_at = <Date 2021-09-15.13:29:13.571>
labels = ['tests', '3.9', '3.10', '3.11']
title = 'multiprocessing tests log: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown'
updated_at = <Date 2021-09-21.18:37:39.101>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-09-21.18:37:39.101>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-21.18:37:39.102>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2021-09-15.13:29:13.571>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 45209
keywords = ['patch']
message_count = 7.0
messages = ['401837', '401866', '401902', '402336', '402341', '402342', '402343']
nosy_count = 4.0
nosy_names = ['vstinner', 'lukasz.langa', 'miss-islington', 'sobolevn']
pr_nums = ['28377', '28499', '28500']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue45209'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    Activity

    1. vstinner commented on Sep 15, 2021

      @vstinner
      vstinner
      on Sep 15, 2021
      MemberAuthor

      0:03:25 load avg: 12.33 [250/427] test_multiprocessing_forkserver passed (...)
      /home/vstinner/python/main/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
      warnings.warn('resource_tracker: There appear to be %d '
      /home/vstinner/python/main/Lib/multiprocessing/resource_tracker.py:237: UserWarning: resource_tracker: '/psm_33506d9a': [Errno 2] No such file or directory: '/psm_33506d9a'
      warnings.warn('resource_tracker: %r: %s' % (name, e))

    2. vstinner
      added
      3.11only security fixes
      testsTests in the Lib/test dir
      on Sep 15, 2021

    3. sobolevn commented on Sep 15, 2021

      @sobolevn
      sobolevn
      on Sep 15, 2021
      Member

      I would like to work on this, if no one has started yet.

    4. sobolevn commented on Sep 15, 2021

      @sobolevn
      sobolevn
      on Sep 15, 2021
      Member

      The problem was in test_shared_memory_cleaned_after_process_termination test.

      » ./python.exe -m test -v test_multiprocessing_forkserver -m test_shared_memory_cleaned_after_process_termination
== CPython 3.11.0a0 (heads/main:09b4ad11f3, Sep 15 2021, 20:50:50) [Clang 11.0.0 (clang-1100.0.33.16)]
== macOS-10.14.6-x86_64-i386-64bit little-endian
== cwd: /Users/sobolev/Desktop/cpython/build/test_python_14874æ
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.98 Run tests sequentially
0:00:00 load avg: 1.98 [1/1] test_multiprocessing_forkserver
test_shared_memory_cleaned_after_process_termination (test.test_multiprocessing_forkserver.WithProcessesTestSharedMemory) ... ok
/Users/sobolev/Desktop/cpython/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/Users/sobolev/Desktop/cpython/Lib/multiprocessing/resource_tracker.py:237: UserWarning: resource_tracker: '/psm_67480e45': [Errno 2] No such file or directory: '/psm_67480e45'
  warnings.warn('resource_tracker: %r: %s' % (name, e))

----------------------------------------------------------------------
Ran 1 test in 0.775s

OK

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1.6 sec
Tests result: SUCCESS

      My patch (location

      cpython/Lib/test/_test_multiprocessing.py

      Line 4179 in 51056b4

      ):

      resource_tracker.unregister(f"/{name}", "shared_memory")

      With this change no warning is generated:

      » ./python.exe -m test -v test_multiprocessing_forkserver -m test_shared_memory_cleaned_after_process_termination
== CPython 3.11.0a0 (heads/main:09b4ad11f3, Sep 15 2021, 20:50:50) [Clang 11.0.0 (clang-1100.0.33.16)]
== macOS-10.14.6-x86_64-i386-64bit little-endian
== cwd: /Users/sobolev/Desktop/cpython/build/test_python_24281æ
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.80 Run tests sequentially
0:00:00 load avg: 1.80 [1/1] test_multiprocessing_forkserver
test_shared_memory_cleaned_after_process_termination (test.test_multiprocessing_forkserver.WithProcessesTestSharedMemory) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.732s

OK

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1.5 sec
Tests result: SUCCESS

      My other highly-related PR where I refactor several SharedMemory tests: #28294

    5. ambv commented on Sep 21, 2021

      @ambv
      ambv
      on Sep 21, 2021
      Contributor

      New changeset f604cf1 by Nikita Sobolev in branch 'main':
      bpo-45209: fix UserWarning: resource_tracker in test_multiprocessing (GH-28377)
      f604cf1

    6. miss-islington commented on Sep 21, 2021

      @miss-islington
      miss-islington
      on Sep 21, 2021
      Contributor

      New changeset 09e5016 by Miss Islington (bot) in branch '3.10':
      bpo-45209: fix UserWarning: resource_tracker in test_multiprocessing (GH-28377)
      09e5016

    7. vstinner commented on Sep 21, 2021

      @vstinner
      vstinner
      on Sep 21, 2021
      MemberAuthor

      New changeset 65268ab by Miss Islington (bot) in branch '3.9':
      bpo-45209: fix UserWarning: resource_tracker in test_multiprocessing (GH-28377) (GH-28500)
      65268ab

    8. vstinner commented on Sep 21, 2021

      @vstinner
      vstinner
      on Sep 21, 2021
      MemberAuthor

      Thanks Nikita Sobolev for the fix.

    9. vstinner
      added
      3.9 (EOL)end of life
      3.10 (EOL)end of life
      on Sep 21, 2021
    10. vstinner
      closed this as completedon Sep 21, 2021
    11. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    12. guglovich commented on Nov 11, 2024

      @guglovich
      guglovich
      on Nov 11, 2024

      On 3.12 on Arch Linux, this error is repeated.

    13. vstinner commented on Nov 12, 2024

      @vstinner
      vstinner
      on Nov 12, 2024
      MemberAuthor

      Please open a new issue wirh a reproducer.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of life3.11only security fixes3.9 (EOL)end of lifetestsTests in the Lib/test dir

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions