Skip to content

Not installed “libgcc_s.so.1” causes exit crash. #87054

Closed
Closed
Not installed “libgcc_s.so.1” causes exit crash.#87054
Assignees
doko42
Labels
3.10 (EOL)end of life3.8 (EOL)end of life3.9 (EOL)end of lifetopic-C-APItype-crashA hard crash of the interpreter, possibly with a core dump

Description

@xxm
xxm
mannequin
opened on Jan 11, 2021
BPO 42888
Nosy @doko42, @vstinner, @tiran, @izbyshev
PRs
  • bpo-42888: Remove PyThread_exit_thread() calls from top-level thread … #24241
    		•
    Superseder
  • bpo-44434: _thread module: Remove redundant PyThread_exit_thread() call to avoid glibc fatal error: libgcc_s.so.1 must be installed for pthread_cancel to work
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/doko42'
closed_at = <Date 2021-06-28.23:29:24.786>
created_at = <Date 2021-01-11.07:40:22.334>
labels = ['expert-C-API', '3.10', '3.8', '3.9', 'type-crash']
title = 'Not installed \xe2\x80\x9clibgcc_s.so.1\xe2\x80\x9d causes exit crash.'
updated_at = <Date 2021-06-28.23:29:24.785>
user = 'https://bugs.python.org/xxm'

    bugs.python.org fields:

    activity = <Date 2021-06-28.23:29:24.785>
actor = 'vstinner'
assignee = 'doko'
closed = True
closed_date = <Date 2021-06-28.23:29:24.786>
closer = 'vstinner'
components = ['C API']
creation = <Date 2021-01-11.07:40:22.334>
creator = 'xxm'
dependencies = []
files = []
hgrepos = []
issue_num = 42888
keywords = ['patch']
message_count = 12.0
messages = ['384798', '384800', '384804', '384810', '384833', '384834', '384890', '385205', '385241', '385242', '385439', '396670']
nosy_count = 5.0
nosy_names = ['doko', 'vstinner', 'christian.heimes', 'izbyshev', 'xxm']
pr_nums = ['24241']
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '44434'
type = 'crash'
url = 'https://bugs.python.org/issue42888'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    Activity

    1. xxm commented on Jan 11, 2021

      @xxm
      xxmmannequin
      on Jan 11, 2021
      MannequinAuthor

      The following thread program will cause Python3.10 parser "core dump" due to missing “libgcc_s.so.1”. "pthread_cancel" cannot work correctly. I am wondering is there any possible to install or link to libgcc_s.so.1 during Python installation?

      ===================================

      import socket
from threading import Thread
class thread(Thread):
      def run(self):
          for res in socket.getaddrinfo('www.google.com',8080):
              sock = socket.socket()
              sock.connect(res[4])

for i in range(2000):
    thread().start()

      ====================================
      Error message:
      ---------------------------------------------------------------

      ........
Exception in thread Thread-1346:
Traceback (most recent call last):
  File "/usr/local/python310/lib/python3.10/threading.py", line 960, in _bootstrap_inner
  File "/usr/local/python310/lib/python3.10/threading.py", line 960, in _bootstrap_inner
Exception in thread Thread-1172:
Traceback (most recent call last):
.......
Exception in thread Thread-1509:
Exception in thread Thread-1510:
libgcc_s.so.1 must be installed for pthread_cancel to work
Exception in thread Thread-1511:
Traceback (most recent call last):
Aborted (core dumped)
    2. xxm
      added
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      3.10 (EOL)end of life
      type-crashA hard crash of the interpreter, possibly with a core dump
      on Jan 11, 2021

    3. tiran commented on Jan 11, 2021

      @tiran
      tiran
      on Jan 11, 2021
      Member

      What's your platform and distribution?

    4. xxm commented on Jan 11, 2021

      @xxm
      xxmmannequin
      on Jan 11, 2021
      MannequinAuthor

      >python310 -V
      Python 3.10.0a2

      >uname -v
      #73~16.04.1-Ubuntu SMP Fri Sep 13 09:56:18 UTC 2019

      Thank you!

    5. tiran commented on Jan 11, 2021

      @tiran
      tiran
      on Jan 11, 2021
      Member

      Please provide more information. What Linux distribution and distro version are you using? How did you install Python?

    6. xxm commented on Jan 11, 2021

      @xxm
      xxmmannequin
      on Jan 11, 2021
      MannequinAuthor

      >uname -a
      Linux xxm-System-Product-Name 4.15.0-64-generic #73~16.04.1-Ubuntu SMP Fri Sep 13 09:56:18 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

      >uname -r
      4.15.0-64-generic

      >lsb_release -a
      No LSB modules are available.
      Distributor ID: Ubuntu
      Description: Ubuntu 16.04.3 LTS
      Release: 16.04
      Codename: xenial

      I download Python from https://www.python.org/downloads/. Then I extract it,run install command ./configure,make,sudo make install. I also try the versions of Python 3.5-3.10 and even cPython on GitHub. The results are the same. But when I try this program on Python 2, the program will not cause core dump. I think Python 2 is initially installed with the system, not by me. So I guess no link is referred to libgcc_s.so.1 when I install new version of Python.

    7. tiran commented on Jan 11, 2021

      @tiran
      tiran
      on Jan 11, 2021
      Member

      It might be a packaging or documentation issue. I'm assiging the ticket to Matthias. He is the Debian and Ubuntu package maintainer.

    8. tiran
      assigned
      doko42
      on Jan 11, 2021

    9. izbyshev commented on Jan 12, 2021

      @izbyshev
      izbyshevmannequin
      on Jan 12, 2021
      Mannequin

      I've encountered this issue too. My use case was a 32-bit Python on a 64-bit CentOS system, and my understanding of the issue was that 64-bit libgcc_s is somehow counted as a "provider" of libgcc_s for 32-bit libc by the package manager, so 32-bit libgcc_s is never installed even when "yum install glibc.i686" is used because everything seems "already OK":

      $ yum deplist glibc

      ...
      dependency: libgcc
      provider: libgcc.x86_64 4.1.2-55.el5
      provider: libgcc.i386 4.1.2-55.el5
      ...

      (The above is for CentOS 5, but at the time I tested 6 and 7 as well, with the same result).

      But I suggest not to dismiss this issue as a packaging bug. Glibc needs libgcc_s for pthread_exit() because it's implemented in terms of pthread_cancel(), and the latter wants to do stack unwinding (probably to cleanup resources for each stack frame). But the code for unwinding lives in libgcc_s. The non-intuitive thing here is that glibc tried to optimize this dependency by dlopen'ing libgcc_s when pthread_cancel() is called instead of making it a startup dependency. Many people consider this a terrible design choice since your program can now fail at an arbitrary moment due to dlopen() failure (and missing libgcc_s is not the only possible reason[1]).

      Since CPython doesn't use pthread_cancel() directly, I propose a simple solution -- just return NULL from thread functions instead of calling pthread_exit(). The last time I looked, pthread_exit() in CPython is only called from top frames of the threads, so a direct return should suffice. If the top-level thread function simply returns, no stack unwinding is needed, so glibc never uses its cancellation machinery.

      I've tested that this solution worked at the time (for 3.8 I think), and the test suite passed. If there is interest in going this way, I can test again.

      [1] https://www.sourceware.org/bugzilla/show_bug.cgi?id=13119

    10. 3 remaining items

    11. terryjreedy
      added
      topic-C-API
      and removed
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      on Jan 15, 2021
    12. terryjreedy
      changed the title [-]Not installed “libgcc_s.so.1” causes parser crash.[/-] [+]Not installed “libgcc_s.so.1” causes exit crash.[/+] on Jan 15, 2021

    13. izbyshev commented on Jan 18, 2021

      @izbyshev
      izbyshevmannequin
      on Jan 18, 2021
      Mannequin

      I've made a PR to remove most calls to pthread_exit().

      @xxm: could you test it in your environment?

    14. xxm commented on Jan 19, 2021

      @xxm
      xxmmannequin
      on Jan 19, 2021
      MannequinAuthor

      No crash is reported anymore. The result is like following:
      -----------------------------------------
      .......
      OSError: [Errno 24] Too many open files
      OSError: [Errno 24] Too many open files
      OSError: [Errno 24] Too many open files
      OSError: [Errno 24] Too many open files
      OSError: [Errno 24] Too many open files
      File "/home/xxm/Downloads/cpython-a4afb55fb226e1debcdaaf80890b790198ba14cc/Lib/socket.py", line 953, in getaddrinfo
      File "/home/xxm/Downloads/cpython-a4afb55fb226e1debcdaaf80890b790198ba14cc/Lib/socket.py", line 953, in getaddrinfo
      OSError: [Errno 24] Too many open files
      File "/home/xxm/Downloads/cpython-a4afb55fb226e1debcdaaf80890b790198ba14cc/Lib/socket.py", line 953, in getaddrinfo
      OSError: [Errno 24] Too many open files
      OSError: [Errno 24] Too many open files
      OSError: [Errno 24] Too many open files
      -----------------------------------------------------------

    15. xxm commented on Jan 19, 2021

      @xxm
      xxmmannequin
      on Jan 19, 2021
      MannequinAuthor

      I think the crash is fixed by this PR. I try other arguments. No crash is reported. Thank you!

    16. izbyshev commented on Jan 21, 2021

      @izbyshev
      izbyshevmannequin
      on Jan 21, 2021
      Mannequin

      Thank you for testing. I've added a NEWS entry to the PR, so it's ready for review by the core devs.

      Note that PyThread_exit_thread() can still be called by daemon threads if they try to take the GIL after Py_Finalize(), and also via C APIs like
      PyEval_RestoreThreads() (see bpo-42969), so, in general, libgcc_s is still required for CPython.

    17. izbyshev
      added
      3.8 (EOL)end of life
      3.9 (EOL)end of life
      on Jan 21, 2021

    18. vstinner commented on Jun 28, 2021

      @vstinner
      vstinner
      on Jun 28, 2021
      Member

      This issue is a duplicate of bpo-44434 which has been fixed.

      I created PR 26943 based on Alexey's PR 24241 to complete my fix.

    19. vstinner
      closed this as completedon Jun 28, 2021
    20. vstinner
      closed this as completedon Jun 28, 2021
    21. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.10 (EOL)end of life3.8 (EOL)end of life3.9 (EOL)end of lifetopic-C-APItype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions