Repository navigation
Upload testthat results to Codecov Test Analytics from R CMD check - #39
Merged
Merged
Conversation
Co-authored-by: Claude Code <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
An active moderate issue remains because a stale or pre-existing junit.xml can be uploaded as current Test Analytics results when setup or test reporting fails.
1 open issue
What changed in this PR
This PR adds optional Codecov Test Analytics uploads for testthat results generated during the existing R CMD check run, without executing the test suite again. The feature remains opt-in through CODECOV_TOKEN and preserves existing coverage and check behavior.
Changes:
- Configures
testthatto emit JUnit XML alongside the normal check reporter. - Uploads JUnit results with
codecov-action@v7, including whenR CMD checkfails and on release branches. - Documents the feature and records the design in ADR 0014.
| File | Description |
|---|---|
| docs/index.html | Updated as part of this pull request. |
| docs/adr/README.md | Updated as part of this pull request. |
| docs/adr/0014-codecov-test-analytics.md | Updated as part of this pull request. |
| bioccheck_usage.md | Updated as part of this pull request. |
| .github/workflows/bioccheck.yml | Updated as part of this pull request. |
Review effort: Lite
Co-authored-by: Claude Code <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
A critical workflow issue can still make R CMD check fail for packages using testthat 2.x, and the ADR status conflicts with the stated separate-approval requirement.
2 open issues
1 resolved since last review
Review effort: Lite
Comment on lines
+167
to
+173
| setHook(packageEvent("testthat", "onLoad"), function(...) { | ||
| options(testthat.default_check_reporter = testthat::MultiReporter$new(list( | ||
| testthat::CheckReporter$new(), | ||
| testthat::JunitReporter$new(file = Sys.getenv("TESTTHAT_JUNIT_FILE")) | ||
| ))) | ||
| }) | ||
| } |
| @@ -0,0 +1,34 @@ | |||
| # 0014. Codecov Test Analytics from the `R CMD check` Run | |||
|
|
|||
| - **Status:** Accepted | |||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



When
CODECOV_TOKENis set, packages tested with testthat now also upload their test results to Codecov Test Analytics. The JUnit XML comes from the existingR CMD checkrun, so the tests are not run again.R CMD check runs
tests/testthat.RwithR --vanilla, which reads no profile except the startup file it copies fromshare/R/tests-startup.R. A step appends anonLoadhook for testthat there, which setstestthat.default_check_reporterto the usualCheckReporterplus aJunitReporter. The output intestthat.Routand the failure on failing tests stay the same. The results are uploaded withcodecov-action@v7andreport_type: test_results, on every branch and also when R CMD check fails. As with coverage, problems here never fail the check.Packages using RUnit or plain
tests/*.Rscripts upload no test results.Details and alternatives are in ADR 0014, which is Proposed and needs separate approval.
Tested locally by extracting the step from the YAML and running a test package through it, serial and with
Config/testthat/parallel: true: correct exit status, unchanged check output, andjunit.xmllisting all tests. Not yet run on GitHub with a token; the canary package has no testthat tests.🤖 Generated with Claude Code