Skip to content

test_math.test_sumprod_stress() takes 5 minutes on debug build #110327

Closed as not planned
Closed as not planned
test_math.test_sumprod_stress() takes 5 minutes on debug build#110327
Labels
testsTests in the Lib/test dir

Description

@vstinner
vstinner
opened on Oct 4, 2023

On the " PPC64LE Fedora Stable Clang 3.x" with Python build by "/configure CC=clang LD=clang --with-pydebug", test_math takes 7 min 39 sec:

make buildbottest 'TESTOPTS=-j2 --junit-xml test-results.xml -j10 --fail-rerun ${BUILDBOT_TESTOPTS}' TESTPYTHONOPTS= TESTTIMEOUT=1200
== CPython 3.13.0a0 (heads/main:269005e784, Oct 3 2023, 23:52:56) [Clang 16.0.6 (Fedora 16.0.6-3.fc38)]
== Linux-5.8.17-200.fc32.ppc64le-ppc64le-with-glibc2.37 little-endian
== Python build: debug
== cwd: /home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/build/test_python_worker_2081694æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
== resources (10): audio, cpu, curses, decimal, gui, largefile, network, subprocess, urlfetch, walltime

(...)
10 slowest tests:
- test_math: 7 min 39 sec
- test.test_multiprocessing_spawn.test_processes: 2 min 49 sec
- test_pathlib: 2 min 36 sec
- test_unparse: 2 min 30 sec
- test_tokenize: 2 min 29 sec
- test.test_multiprocessing_forkserver.test_processes: 2 min 22 sec
- test_capi: 1 min 52 sec
- test_cppext: 1 min 42 sec
- test_unicodedata: 1 min 39 sec
- test_tarfile: 1 min 33 sec

Can we split test_math into a test package made of multiple tests to run them in parallel?

Should we remove/skip some slow tests? Buildbot uses --slow-ci which enables -u all resources.

Activity

  1. vstinner commented on Oct 4, 2023

    @vstinner
    vstinner
    on Oct 4, 2023
    MemberAuthor

    Timings as JUnit XML of ./python -m test test_math -v --junit-xml=results-all.xml -u all: results-all.xml.txt (5 min 28 sec).

  2. AlexWaygood
    added
    testsTests in the Lib/test dir
    on Oct 4, 2023

  3. vstinner commented on Oct 5, 2023

    @vstinner
    vstinner
    on Oct 5, 2023
    MemberAuthor

    Timings as JUnit XML of ./python -m test test_math -v --junit-xml=results-all.xml -u all: results-all.xml.txt (5 min 28 sec).

    test_math.test_sumprod_stress() alone takes 5 min 13 sec. The test was added by issue GH-100485: commit 47b9f83.

  4. vstinner commented on Oct 5, 2023

    @vstinner
    vstinner
    on Oct 5, 2023
    MemberAuthor

    test_math.test_sumprod_stress() alone takes 5 min 13 sec

    This test uses 9 pools of lengths: [10, 9, 9, 11, 10, 9, 8, 5, 8]. It uses 4 nested loops for a total of 5,969,664 iterations.

    IMO that's excessive and it should be possible to minimize the test to a few cases. I don't think that there is a need to test 5,969,664 cases.

  5. vstinner commented on Oct 5, 2023

    @vstinner
    vstinner
    on Oct 5, 2023
    MemberAuthor

    @rhettinger @serhiy-storchaka: Can you please have a look at test_math.test_sumprod_stress() and see if we can test a few cases, rather than testing 5,969,664 cases?

    On my laptop with Python built with gcc -O0, ./python -m test test_math -u all takes 10 min 50 sec :-(

  6. vstinner commented on Oct 5, 2023

    @vstinner
    vstinner
    on Oct 5, 2023
    MemberAuthor
    • On Python built with gcc -O3 (./configure), ./python -m test test_math -u all -v -m test_sumprod_stress takes 25 seconds.
    • On Python built with gcc -Og (./configure --with-pydebug), ./python -m test test_math -u all -v -m test_sumprod_stress takes 1 min 49 sec.

  7. serhiy-storchaka commented on Oct 5, 2023

    @serhiy-storchaka
    serhiy-storchaka
    on Oct 5, 2023
    Member

    Sorry, I don't know exactly what it is testing and what can be cut out without losing the value of the test.

  8. skirpichev commented on Oct 6, 2023

    @skirpichev
    skirpichev
    on Oct 6, 2023
    Member

    if we can test a few cases, rather than testing 5,969,664 cases?

    @vstinner, test_sumprod_stress() should cover some lines, mentioned in #110000 (e.g. overflows for big integers), but I think at least in this function we could test only "exact" floats/ints, while mixed inputs could go to the testSumProd().

  9. vstinner commented on Dec 20, 2023

    @vstinner
    vstinner
    on Dec 20, 2023
    MemberAuthor

    Well, apparently, other people don't care much, so let's leave it like that for now. If someone wants to propose a change, please open a new issue.

  10. vstinner
    closed this as completedon Dec 20, 2023
  11. vstinner
    changed the title [-]Running test_math on PPC64LE Fedora Stable Clang 3.x takes 7 min 39 sec[/-] [+]test_math.test_sumprod_stress() takes 5 minutes on debug build[/+] on Apr 11, 2024
  12. vstinner
    closed this as not plannedon Apr 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dir

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions