Skip to content

Upload testthat results to Codecov Test Analytics from R CMD check - #39

Merged
lwaldron merged 3 commits into
mainfrom
codecov-test-analytics
Nov 5, 2026
Merged

lwaldron merged 3 commits into
mainfrom
codecov-test-analytics

Conversation

@lwaldron

@lwaldron lwaldron commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

When CODECOV_TOKEN is set, packages tested with testthat now also upload their test results to Codecov Test Analytics. The JUnit XML comes from the existing R CMD check run, so the tests are not run again.

R CMD check runs tests/testthat.R with R --vanilla, which reads no profile except the startup file it copies from share/R/tests-startup.R. A step appends an onLoad hook for testthat there, which sets testthat.default_check_reporter to the usual CheckReporter plus a JunitReporter. The output in testthat.Rout and the failure on failing tests stay the same. The results are uploaded with codecov-action@v7 and report_type: test_results, on every branch and also when R CMD check fails. As with coverage, problems here never fail the check.

Packages using RUnit or plain tests/*.R scripts upload no test results.

Details and alternatives are in ADR 0014, which is Proposed and needs separate approval.

Tested locally by extracting the step from the YAML and running a test package through it, serial and with Config/testthat/parallel: true: correct exit status, unchanged check output, and junit.xml listing all tests. Not yet run on GitHub with a token; the canary package has no testthat tests.

🤖 Generated with Claude Code

Co-authored-by: Claude Code <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An active moderate issue remains because a stale or pre-existing junit.xml can be uploaded as current Test Analytics results when setup or test reporting fails.

1 open issue
What changed in this PR

This PR adds optional Codecov Test Analytics uploads for testthat results generated during the existing R CMD check run, without executing the test suite again. The feature remains opt-in through CODECOV_TOKEN and preserves existing coverage and check behavior.

Changes:

  • Configures testthat to emit JUnit XML alongside the normal check reporter.
  • Uploads JUnit results with codecov-action@v7, including when R CMD check fails and on release branches.
  • Documents the feature and records the design in ADR 0014.
File Description
docs/​index.html Updated as part of this pull request.
docs/​adr/​README.md Updated as part of this pull request.
docs/​adr/​0014-codecov-test-analytics.md Updated as part of this pull request.
bioccheck_usage.md Updated as part of this pull request.
.github/​workflows/​bioccheck.yml Updated as part of this pull request.

Review effort: Lite

Comment thread .github/workflows/bioccheck.yml Resolved
Co-authored-by: Claude Code <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical workflow issue can still make R CMD check fail for packages using testthat 2.x, and the ADR status conflicts with the stated separate-approval requirement.

2 open issues
1 resolved since last review

Review effort: Lite

Comment on lines +167 to +173
setHook(packageEvent("testthat", "onLoad"), function(...) {
options(testthat.default_check_reporter = testthat::MultiReporter$new(list(
testthat::CheckReporter$new(),
testthat::JunitReporter$new(file = Sys.getenv("TESTTHAT_JUNIT_FILE"))
)))
})
}
@@ -0,0 +1,34 @@
# 0014. Codecov Test Analytics from the `R CMD check` Run

- **Status:** Accepted
@lwaldron
lwaldron merged commit 43aac86 into main Nov 5, 2026
3 checks passed
@lwaldron
lwaldron deleted the codecov-test-analytics branch November 5, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants