Skip to content

test_os.TimerfdTests is flaky #126112

Description

@pepoluan

Bug report

Bug description:

When compiling dev-lang/python-3.13.0 on Gentoo, the following tests failed:

======================================================================
FAIL: test_timerfd_interval (test.test_os.TimerfdTests.test_timerfd_interval)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.13.0/work/Python-3.13.0/Lib/test/test_os.py", line 4238, in test_timerfd_interval
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 4 != 1

======================================================================
FAIL: test_timerfd_ns_select (test.test_os.TimerfdTests.test_timerfd_ns_select)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.13.0/work/Python-3.13.0/Lib/test/test_os.py", line 4457, in test_timerfd_ns_select
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 1

======================================================================
FAIL: test_timerfd_poll (test.test_os.TimerfdTests.test_timerfd_poll)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.13.0/work/Python-3.13.0/Lib/test/test_os.py", line 4349, in test_timerfd_poll
    self.check_timerfd_poll(False)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
  File "/var/tmp/portage/dev-lang/python-3.13.0/work/Python-3.13.0/Lib/test/test_os.py", line 4337, in check_timerfd_poll
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 1

======================================================================
FAIL: test_timerfd_select (test.test_os.TimerfdTests.test_timerfd_select)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/portage/dev-lang/python-3.13.0/work/Python-3.13.0/Lib/test/test_os.py", line 4292, in test_timerfd_select
    self.assertEqual(self.read_count_signaled(fd), 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 5 != 1

----------------------------------------------------------------------
Ran 366 tests in 32.208s

FAILED (failures=4, skipped=62)
test test_os failed

USE flags used are:

USE="ensurepip gdbm ncurses pgo readline sqlite ssl -bluetooth -build -debug -examples (-jit) -libedit* -test -tk -valgrind -verify-sig"

which resulted in the following ./configure invocation:

./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --datarootdir=/usr/share --docdir=/usr/share/doc/python-3.13.0 --htmldir=/usr/share/doc/python-3.13.0/html --libdir=/usr/lib64 ac_cv_header_stropts_h=no --enable-shared --without-static-libpython --enable-ipv6 --infodir=${prefix}/share/info --mandir=${prefix}/share/man --with-computed-gotos --with-dbmliborder=gdbm --with-libc= --enable-loadable-sqlite-extensions --without-ensurepip --without-lto --with-system-expat --with-system-libmpdec --with-platlibdir=lib --with-pkg-config=yes --with-wheel-pkg-dir=/usr/lib/python/ensurepip --enable-gil --without-assertions --disable-experimental-jit --enable-optimizations --with-readline=readline --without-valgrind --with-lto

CPython versions tested on:

3.13.0, 3.13.1

Operating systems tested on:

Linux (Gentoo)

Linked PRs

Activity

  1. pepoluan commented on Dec 13, 2024

    @pepoluan
    ContributorAuthor

    I think this bug is similar to #124873 , though that one was on Android, while this one is on a (heavily-customized) Gentoo Linux.

  2. pepoluan commented on Dec 13, 2024

    @pepoluan
    ContributorAuthor

    Update:

    Upgrading to 3.13.1 still resulted in a similar error:

    
    ======================================================================
    FAIL: test_timerfd_TFD_TIMER_ABSTIME (test.test_os.TimerfdTests.test_timerfd_TFD_TIMER_ABSTIME)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/var/tmp/portage/dev-lang/python-3.13.1/work/Python-3.13.1/Lib/test/test_os.py", line 4266, in test_timerfd_TFD_TIMER_ABSTIME
        self.assertAlmostEqual(next_expiration, offset, places=self.CLOCK_RES_PLACES)
        ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: 0.99945497 != 1 within 3 places (0.0005450299999999464 difference)
    
    ======================================================================
    FAIL: test_timerfd_ns_poll (test.test_os.TimerfdTests.test_timerfd_ns_poll)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/var/tmp/portage/dev-lang/python-3.13.1/work/Python-3.13.1/Lib/test/test_os.py", line 4353, in test_timerfd_ns_poll
        self.check_timerfd_poll(True)
        ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
      File "/var/tmp/portage/dev-lang/python-3.13.1/work/Python-3.13.1/Lib/test/test_os.py", line 4338, in check_timerfd_poll
        self.assertEqual(self.read_count_signaled(fd), 1)
        ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: 4 != 1
    
    ----------------------------------------------------------------------
    Ran 366 tests in 34.558s
    
    FAILED (failures=2, skipped=62)
    test test_os failed
    
  3. mhsmith commented on Dec 13, 2024

    @mhsmith
    Member

    Interesting – I couldn't see how the test could fail this way unless the process had been suspended for 1/8 of a second. This seemed unlikely even on an Android emulator, but possible given Android's customized process scheduler. But if it's failing even on Linux, there's probably some other cause.

  4. changed the title [-]test_os.TimerfdTests fail[/-] [+]test_os.TimerfdTests fail on Gentoo[/+] on Dec 13, 2024
  5. rossburton commented on Feb 24, 2025

    @rossburton
    Contributor

    We've seen this in the Yocto Project when running the python tests inside a qemu on a heavily loaded system: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15743.

  6. mhsmith commented on Feb 24, 2025

    @mhsmith
    Member

    The Yocto failure isn't in the same test, it's in a different timerfd test which is much more timing-sensitive. See discussion in #130508.

  7. mhsmith commented on Feb 24, 2025

    @mhsmith
    Member

    I notice that most of the timerfd tests create an FD and never close it. This shouldn't directly lead to the failure above, because every test uses a separate FD, but it would still be worth cleaning up.

  8. picnixz commented on Feb 25, 2025

    @picnixz
    Member

    FTR, it also failed on AMD Debian: #129797 (comment)

  9. colesbury commented on Mar 12, 2025

    @colesbury
    Contributor

    I saw this locally on Ubuntu 22.04 x86-64:

    ======================================================================
    FAIL: test_timerfd_TFD_TIMER_ABSTIME (test.test_os.TimerfdTests.test_timerfd_TFD_TIMER_ABSTIME)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/raid/sgross/cpython/Lib/test/test_os.py", line 4419, in test_timerfd_TFD_TIMER_ABSTIME
        self.assertAlmostEqual(next_expiration, offset, places=self.CLOCK_RES_PLACES)
        ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: 0.999380538 != 1 within 3 places (0.0006194619999999595 difference)
    
    ----------------------------------------------------------------------
    
  10. furkanonder commented on Mar 14, 2025

    @furkanonder
    Contributor

    @colesbury It also fails on NetBSD, see #131266

  11. changed the title [-]test_os.TimerfdTests fail on Gentoo[/-] [+]test_os.TimerfdTests is flaky[/+] on Mar 14, 2025
  12. vstinner commented on May 1, 2025

    @vstinner
    Member

    test_time fails randomly on s390x Fedora Stable LTO + PGO 3.x.

    Example: https://buildbot.python.org/#/builders/1627/builds/314

    FAIL: test_timerfd_ns_poll (test.test_os.TimerfdTests.test_timerfd_ns_poll)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-s390x.lto-pgo/build/Lib/test/test_os.py", line 4512, in test_timerfd_ns_poll
        self.check_timerfd_poll(True)
        ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
      File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-s390x.lto-pgo/build/Lib/test/test_os.py", line 4497, in check_timerfd_poll
        self.assertEqual(self.read_count_signaled(fd), 1)
        ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: 2 != 1
    
  13. StanFromIreland commented on Jun 18, 2025

    @StanFromIreland
    Member

    This occurs every now and then on the 3.14 (which is weird, since it should also effect others) branch on ARM64 Raspbian buildbot:

    full log

    ======================================================================
    FAIL: test_timerfd_interval (test.test_os.TimerfdTests.test_timerfd_interval)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/stan/buildarea/3.14.stan-raspbian.nondebug/build/Lib/test/test_os.py", line 4402, in test_timerfd_interval
        self.assertGreater(t, total_time - self.CLOCK_RES)
        ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: 1.9921359503641725 not greater than 1.999
    

    full log

    ======================================================================
    FAIL: test_timerfd_initval (test.test_os.TimerfdTests.test_timerfd_initval)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/stan/buildarea/3.14.stan-raspbian.nondebug/build/Lib/test/test_os.py", line 4335, in test_timerfd_initval
        self.assertAlmostEqual(next_expiration, initial_expiration, places=self.CLOCK_RES_PLACES)
        ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: 0.24579456700000002 != 0.25 within 3 places (0.004205432999999981 difference)
    ----------------------------------------------------------------------
    

    Maybe we should lower the default resolution to 2 places?

  14. added 2 commits that reference this issue on Jun 18, 2025
  15. vstinner commented on Jun 18, 2025

    @vstinner
    Member

    Maybe we should lower the default resolution to 2 places?

    It makes sense. I wrote #135681 to do exactly that.

  16. added a commit that references this issue on Jun 19, 2025
  17. added 2 commits that reference this issue on Jun 19, 2025
  18. added 2 commits that reference this issue on Jun 19, 2025
  19. added a commit that references this issue on Jun 19, 2025
  20. added a commit that references this issue on Jul 12, 2025
  21. added a commit that references this issue on Aug 4, 2025
  22. added a commit that references this issue on Aug 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions