Current Situation
There are a number of tests in the suite that are very flaky (usually requiring 2-3 runs to pass). For new contributors this is likely to be confusing since they'll be unsure if those failures are related to their changes. Unfortunately the problems are hard to reproduce since running the local tests rarely, if ever, fails at the rate seen in CI/CD. I've also noticed that there are times when the CI/CD will pass on the first go, while on others it will simply refuse to pass.
Proposed Actions
It's likely that these flaky tests are missing some sort of locking mechanism to ensure that actions occur in the correct order (or at least that's been my experience in the past).
Current Situation
There are a number of tests in the suite that are very flaky (usually requiring 2-3 runs to pass). For new contributors this is likely to be confusing since they'll be unsure if those failures are related to their changes. Unfortunately the problems are hard to reproduce since running the local tests rarely, if ever, fails at the rate seen in CI/CD. I've also noticed that there are times when the CI/CD will pass on the first go, while on others it will simply refuse to pass.
Proposed Actions
It's likely that these flaky tests are missing some sort of locking mechanism to ensure that actions occur in the correct order (or at least that's been my experience in the past).