Repository navigation
PGO tests fail on 3.11 and 3.12 #111929
Copy link
Copy link
Closed
Labels
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
Bug report
Bug description:
Recent PGO tests on Python 3.11 and 3.12 result in failure.
For example, Linux Buildbots' logs (compile section) are like:
Using random seed: 2130237982
0:00:00 load avg: 1.75 Run 44 tests sequentially (timeout: 20 min)
0:00:00 load avg: 1.75 [ 1/44] test_array
0:00:01 load avg: 1.75 [ 2/44] test_base64
0:00:02 load avg: 1.75 [ 3/44] test_binascii
0:00:02 load avg: 1.75 [ 4/44] test_binop
0:00:02 load avg: 1.75 [ 5/44] test_bisect
0:00:03 load avg: 1.75 [ 6/44] test_bytes
0:00:15 load avg: 1.58 [ 7/44] test_bz2
0:00:16 load avg: 1.58 [ 8/44] test_cmath
0:00:16 load avg: 1.58 [ 9/44] test_codecs
0:00:19 load avg: 1.54 [10/44] test_collections
0:00:22 load avg: 1.54 [11/44] test_complex
0:00:22 load avg: 1.54 [12/44] test_dataclasses
0:00:23 load avg: 1.49 [13/44] test_datetime
0:00:32 load avg: 1.45 [14/44] test_decimal
0:00:42 load avg: 1.38 [15/44] test_difflib
0:00:45 load avg: 1.35 [16/44] test_embed
0:01:00 load avg: 1.12 [17/44] test_float
0:01:00 load avg: 1.12 [18/44] test_fstring
0:01:03 load avg: 1.12 [19/44] test_functools
0:01:04 load avg: 1.11 [20/44] test_generators
0:01:04 load avg: 1.11 [21/44] test_hashlib
0:01:06 load avg: 1.11 [22/44] test_heapq
0:01:08 load avg: 1.11 [23/44] test_int
0:01:09 load avg: 1.10 [24/44] test_itertools
0:01:20 load avg: 1.09 [25/44] test_json
0:01:25 load avg: 1.00 [26/44] test_long
0:01:33 load avg: 1.00 [27/44] test_lzma
0:01:34 load avg: 1.00 [28/44] test_math
0:01:42 load avg: 1.00 [29/44] test_memoryview
0:01:43 load avg: 1.00 [30/44] test_operator
0:01:43 load avg: 1.00 [31/44] test_ordered_dict
0:01:45 load avg: 1.00 [32/44] test_patma
0:01:46 load avg: 1.00 [33/44] test_pickle
0:02:01 load avg: 1.00 [34/44] test_pprint
0:02:02 load avg: 1.00 [35/44] test_re
0:02:04 load avg: 1.00 [36/44] test_set
0:02:18 load avg: 1.00 [37/44] test_sqlite3
0:02:19 load avg: 1.00 [38/44] test_statistics
0:02:25 load avg: 1.00 [39/44] test_str
0:02:25 load avg: 1.00 [40/44] test_struct -- test_str failed (uncaught exception)
0:02:26 load avg: 1.00 [41/44] test_tabnanny
0:02:27 load avg: 1.00 [42/44] test_time
0:02:30 load avg: 0.92 [43/44] test_xml_etree
0:02:31 load avg: 0.92 [44/44] test_xml_etree_c
Total duration: 2 min 33 sec
Total tests: run=8,699 skipped=193
Total test files: run=44/44 failed=1
Result: FAILURE
Using random seed: 2899545713
0:00:00 load avg: 2.37 Run 44 tests sequentially (timeout: 20 min)
0:00:00 load avg: 2.37 [ 1/44] test_array
0:00:02 load avg: 2.37 [ 2/44] test_base64
0:00:02 load avg: 2.37 [ 3/44] test_binascii
0:00:03 load avg: 2.37 [ 4/44] test_binop
0:00:03 load avg: 2.37 [ 5/44] test_bisect
0:00:03 load avg: 2.37 [ 6/44] test_bytes
0:00:12 load avg: 2.16 [ 7/44] test_bz2
0:00:13 load avg: 2.16 [ 8/44] test_cmath
0:00:14 load avg: 2.16 [ 9/44] test_codecs
0:00:14 load avg: 2.16 [10/44] test_collections -- test_codecs failed (uncaught exception)
0:00:14 load avg: 2.16 [11/44] test_complex -- test_collections failed (uncaught exception)
0:00:14 load avg: 2.16 [12/44] test_dataclasses -- test_complex failed (uncaught exception)
0:00:14 load avg: 2.16 [13/44] test_datetime -- test_dataclasses failed (uncaught exception)
0:00:14 load avg: 2.16 [14/44] test_decimal -- test_datetime failed (uncaught exception)
0:00:14 load avg: 2.16 [15/44] test_difflib -- test_decimal failed (uncaught exception)
0:00:14 load avg: 2.16 [16/44] test_embed -- test_difflib failed (uncaught exception)
0:00:14 load avg: 2.16 [17/44] test_float -- test_embed failed (uncaught exception)
0:00:14 load avg: 2.16 [18/44] test_fstring -- test_float failed (uncaught exception)
0:00:14 load avg: 2.16 [19/44] test_functools -- test_fstring failed (uncaught exception)
0:00:14 load avg: 2.16 [20/44] test_generators -- test_functools failed (uncaught exception)
0:00:14 load avg: 2.16 [21/44] test_hashlib -- test_generators failed (uncaught exception)
0:00:14 load avg: 2.16 [22/44] test_heapq -- test_hashlib failed (uncaught exception)
0:00:14 load avg: 2.16 [23/44] test_int -- test_heapq failed (uncaught exception)
0:00:14 load avg: 2.16 [24/44] test_itertools -- test_int failed (uncaught exception)
0:00:14 load avg: 2.16 [25/44] test_json -- test_itertools failed (uncaught exception)
0:00:14 load avg: 2.16 [26/44] test_long -- test_json failed (uncaught exception)
0:00:14 load avg: 2.16 [27/44] test_lzma -- test_long failed (uncaught exception)
0:00:14 load avg: 2.16 [28/44] test_math -- test_lzma failed (uncaught exception)
0:00:14 load avg: 2.16 [29/44] test_memoryview -- test_math failed (uncaught exception)
0:00:14 load avg: 2.16 [30/44] test_operator -- test_memoryview failed (uncaught exception)
0:00:14 load avg: 2.16 [31/44] test_ordered_dict -- test_operator failed (uncaught exception)
0:00:14 load avg: 2.16 [32/44] test_patma -- test_ordered_dict failed (uncaught exception)
0:00:14 load avg: 2.16 [33/44] test_pickle -- test_patma failed (uncaught exception)
0:00:14 load avg: 2.16 [34/44] test_pprint -- test_pickle failed (uncaught exception)
0:00:14 load avg: 2.16 [35/44] test_re -- test_pprint failed (uncaught exception)
0:00:14 load avg: 2.16 [36/44] test_set -- test_re failed (uncaught exception)
0:00:14 load avg: 2.16 [37/44] test_sqlite3 -- test_set failed (uncaught exception)
0:00:14 load avg: 2.16 [38/44] test_statistics -- test_sqlite3 failed (uncaught exception)
0:00:14 load avg: 2.16 [39/44] test_str -- test_statistics failed (uncaught exception)
0:00:14 load avg: 2.16 [40/44] test_struct -- test_str failed (uncaught exception)
0:00:14 load avg: 2.16 [41/44] test_tabnanny -- test_struct failed (uncaught exception)
0:00:14 load avg: 2.16 [42/44] test_time -- test_tabnanny failed (uncaught exception)
0:00:14 load avg: 2.16 [43/44] test_xml_etree -- test_time failed (uncaught exception)
0:00:14 load avg: 2.16 [44/44] test_xml_etree_c -- test_xml_etree failed (uncaught exception)
test_xml_etree_c failed (uncaught exception)
Total duration: 14.2 sec
Total tests: run=1,388 skipped=9
Total test files: run=44/44 failed=36
Result: FAILURE
Seems related to issue gh-110276, gh-110756.
cc: @vstinner @AlexWaygood
CPython versions tested on:
3.12
Operating systems tested on:
Windows
Linked PRs
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Activity
pythongh-111929: Fix regrtest --pgo: test_str => test_unicode
gh-111929: Fix regrtest --pgo: test_str => test_unicode (#111938)
pythongh-111929: Fix regrtest --pgo: test_str => test_unicode (python…
vstinner commentedon Nov 10, 2023
I'm working on fixes.
AlexWaygood commentedon Nov 10, 2023
How do we ensure this isn't repeated the next time we try to sync libregrtest in the backport branches with the code on the main branch?
[3.11] gh-111929: Fix regrtest --pgo: test_str => test_unicode (GH-11…
vstinner commentedon Nov 10, 2023
I'm considering to backport the Makefile change which treats a failure... as a failure: don't ignore the failure.
vstinner commentedon Nov 10, 2023
I created #111950 for that.
[3.11] gh-111929: Fix regrtest clear_caches() (#111949)
vstinner commentedon Nov 15, 2023
Thanks @neonene for your bug report!