--experimental-test-coverage empty since 23.5.0

Version

v24.1.0

Platform

Darwin 24.4.0 Darwin Kernel Version 24.4.0: Fri Apr 11 18:34:14 PDT 2025; root:xnu-11417.101.15~117/RELEASE_ARM64_T8122 arm64

Subsystem

test

What steps will reproduce the bug?

import { test } from 'node:test'

console.log(process.versions.node)

test('foo', () => {
  if (false) {
    throw new Error('Failed')
  } else {
    // pass
  }
})

% node --experimental-test-coverage --test i.js    
24.1.0
✔ foo (0.502334ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 57.841208
ℹ start of coverage report
ℹ ----------------------------------------------------------
ℹ file      | line % | branch % | funcs % | uncovered lines
ℹ ----------------------------------------------------------
ℹ ----------------------------------------------------------
ℹ all files | 100.00 |   100.00 |  100.00 | 
ℹ ----------------------------------------------------------
ℹ end of coverage report

Since 23.5.0:

% nvm use 23.5
Now using node v23.5.0 (npm v10.9.2)
% node --experimental-test-coverage --test i.js
23.5.0
✔ foo (0.330042ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 64.134167
ℹ start of coverage report
ℹ -----------------------------------------------------
ℹ file | line % | branch % | funcs % | uncovered lines
ℹ -----------------------------------------------------
ℹ -----------------------------------------------------
ℹ …les | 100.00 |   100.00 |  100.00 | 
ℹ -----------------------------------------------------
ℹ end of coverage report

23.4.0 was fine:

% nvm use 23.4                                 
Now using node v23.4.0 (npm v10.9.2)
% node --experimental-test-coverage --test i.js
23.4.0
✔ foo (0.326084ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 62.499375
ℹ start of coverage report
ℹ -------------------------------------------------------
ℹ file   | line % | branch % | funcs % | uncovered lines
ℹ -------------------------------------------------------
ℹ i.js   |  91.67 |    66.67 |  100.00 | 7
ℹ -------------------------------------------------------
ℹ …files |  91.67 |    66.67 |  100.00 | 
ℹ -------------------------------------------------------
ℹ end of coverage report

How often does it reproduce? Is there a required condition?

Every time

What is the expected behavior? Why is that the expected behavior?

Output on 23.4.0 (see above) was fine
Non-empty coverage report, reporting line 7 in the above example as uncovered

What do you see instead?

Empty coverage

Additional information

No response

Ah, #56060
But --test-coverage-include i.js doesn't help this, so I was confused

No, this still is confusing

In this setup, i0.js isn't a test file, but is ignored:

i.js:

import { test } from 'node:test'
import { foo } from './i0.js'

console.log(process.versions.node)

test('foo', () => {
  foo(2)
})

i0.js:

export function foo(arg) {
  if (!arg) {
    throw new Error('Failed')
  } else {
    return arg * 2
  }
}

Before 23.5:

23.4.0
✔ foo (0.345917ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 74.277709
ℹ start of coverage report
ℹ --------------------------------------------------------
ℹ file    | line % | branch % | funcs % | uncovered lines
ℹ --------------------------------------------------------
ℹ i.js    | 100.00 |   100.00 |  100.00 | 
ℹ i0.js   |  85.71 |    66.67 |  100.00 | 3
ℹ --------------------------------------------------------
ℹ … files |  93.33 |    80.00 |  100.00 | 
ℹ --------------------------------------------------------
ℹ end of coverage report

Since 23.5:

23.5.0
✔ foo (6.384833ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 164.013708
ℹ start of coverage report
ℹ -----------------------------------------------------
ℹ file | line % | branch % | funcs % | uncovered lines
ℹ -----------------------------------------------------
ℹ -----------------------------------------------------
ℹ …les | 100.00 |   100.00 |  100.00 | 
ℹ -----------------------------------------------------
ℹ end of coverage report

--test-coverage-include=i0.js doesn't help
--test-coverage-exclude='' doesn't work
--test-coverage-exclude='some-invalid-pattern' helps, but is... strange

... ok, this apparently excludes files based on their global location, and if the project is inside a subdir of a .../test/... dir, everything is excluded

which looks confusing, but not very significant to care

@ChALkeR

Hey @ChALkeR, as per documentation the latest versions are excluding from the coverage the matching test files:

"By default all the matching test files are excluded from the coverage report. Specifying this option will override the default behavior." (https://nodejs.org/docs/latest/api/cli.html#--test-coverage-exclude)

@pmarchini

@pmarchini I see that, the issue was that those were not test files
E.g. if an os user is named test, everything in their projects would likely be excluded, as this operates on global paths, not on relative paths

This is unfortunate, but (as I already said) not very significant

@ChALkeR

Sorry @ChALkeR, I think I didn’t express myself clearly.

You're right about the match pattern.

At the moment, we're excluding, by default, everything that matches the inclusion patterns, which may need clarification.

It might be worth checking whether the documentation makes this clear, since what I shared definitely isn't clear enough

Or perhaps matching should instead be performed on paths relative to the current project root / closest package.json file / cwd as fallback
Or use the files that are matched by node --test

node --test is also documented to include tests matching inclusion patterns, but it does not treat global paths as matches

I would have expected this to exclude the exact same files that are matched by node --test

I think this is a bug.

@targos

I think this is a bug.

Hey @targos, IMHO, I think it's working as expected, even though the documentation could be more detailed.

Rather than a bug, I think we should discuss potential alternative solutions!

@ChALkeR

I would have expected this to exclude the exact same files that are matched by node --test

@ChALkeR if I'm not mistaken we discussed this in another issue (which I can't find at the moment).
I agree; that seems reasonable to me.

If I'm not mistaken, the reason we avoided this approach was due to the invasiveness of the required changes.
Nonetheless, we could discuss it again.

Not reproducible on Node.js git build.

./node/node --experimental-test-coverage --test i.js
25.0.0-pre
✔ foo (0.3185ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 54.209292
ℹ start of coverage report
ℹ ----------------------------------------------------------
ℹ file      | line % | branch % | funcs % | uncovered lines
ℹ ----------------------------------------------------------
ℹ i.js      |  90.91 |    66.67 |  100.00 | 7
ℹ ----------------------------------------------------------
ℹ all files |  90.91 |    66.67 |  100.00 |
ℹ ----------------------------------------------------------
ℹ end of coverage report

@ChaseKnowlden

@ChaseKnowlden what's pwd?

test: add fixtures for cwd-relative coverage exclusion

test: add test file for cwd-relative coverage exclusion fixture

test: add regression cases for cwd-relative coverage exclusion

test_runner: match built-in coverage exclusions against cwd-relative …

test: cover nested node_modules exclusion

test_runner: bound built-in coverage exclusions to cwd-relative paths

test_runner: match coverage globs against cwd-relative paths only

Navigation Menu

Repository navigation

--experimental-test-coverage empty since 23.5.0 #58654

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior? Why is that the expected behavior?

What do you see instead?

Additional information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

--experimental-test-coverage empty since 23.5.0 #58654

Description

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior? Why is that the expected behavior?

What do you see instead?

Additional information

Activity

ChALkeR commented on Jun 10, 2025

ChALkeR commented on Jun 10, 2025

ChALkeR commented on Jun 10, 2025

pmarchini commented on Jun 10, 2025

ChALkeR commented on Jun 10, 2025

pmarchini commented on Jun 10, 2025

ChALkeR commented on Jun 10, 2025

targos commented on Jun 11, 2025

pmarchini commented on Jun 11, 2025

pmarchini commented on Jun 11, 2025

ChaseKnowlden commented on Jun 11, 2025

ChALkeR commented on Jun 11, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions