Repository navigation
libregrtest should always set random.seed #110171
Description
Feature or enhancement
While working on #110160 I've noticed that it is rather hard to reproduce random test failures. So, I want to propose a new feature / fix for that.
First of all, there's existing prior work of @vstinner who added --randseed flag.
Right now it is used together with -r to randomize test order and seeding random.
I propose to:
- Split
--randseedinto two options: test order randomization andrandom.seedusage - Let's keep
-ras-is - Let's add
--no-use-randseedflag to disable settingrandom.seed, let's always seed random by default - Always print information about current random seed to be able to reuse it in the next runs
Example:
» ./python.exe -m test test_regrtest
Using random seed 65906482
0:00:00 load avg: 1.55 Run 1 test sequentially
0:00:00 load avg: 1.55 [1/1] test_regrtest
== Tests result: SUCCESS ==
1 test OK.
Total duration: 12.7 sec
Total tests: run=102 skipped=2
Total test files: run=1/1
Result: SUCCESS
Basically, this is how https://github.com/pytest-dev/pytest-randomly works.
I have a PR ready.
Activity
pythongh-110171: `libregrtest` always sets `random.seed` unless `--no…
pythongh-110171: `libregrtest` always sets `random.seed` unless `--no…
terryjreedy commentedon Oct 1, 2023
I think that the seed should be random but recorded so that a run can be reproduced.
sobolevn commentedon Oct 1, 2023
Yes 👍
@terryjreedy please, check my PR.
vstinner commentedon Oct 1, 2023
I had a similar idea.
I would prefer that
--randomizekeeps its current behavior: randomize the list of tests. It has a significant impact when tests are run sequentially.As an user, it's hard to understand this option. I don't understand the use case neither. Is it to increase the entropy of the initial seed? Recently, I increased the rand seed range to make it 10x larger :-)
I suggest to not add such confusing option. Just always set a random seed and display it.
gh-110171: `libregrtest` always sets `random.seed` (#110172)
sobolevn commentedon Oct 4, 2023
I've checked all other
Random()instances inLib/testnone of them need extra seeding.Most of them are in
test_random, which handles seeds correctly!vstinner commentedon Oct 4, 2023
Thanks for checking.
pythongh-110171: `libregrtest` always sets `random.seed` (python#110172)