nodejs
--experimental-test-coverage empty since 23.5.0
Version
v24.1.0
Platform
Darwin macbook-air.taile5a6b.ts.net 24.4.0 Darwin Kernel Version 24.4.0: Fri Apr 11 18:34:14 PDT 2025; root:xnu-11417.101.15~117/RELEASE_ARM64_T8122 arm64
Subsystem
test
What steps will reproduce the bug?
import { test } from 'node:test'
console.log(process.versions.node)
test('foo', () => {
if (false) {
throw new Error('Failed')
} else {
// pass
}
})
% node --experimental-test-coverage --test i.js
24.1.0
✔ foo (0.502334ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 57.841208
ℹ start of coverage report
ℹ ----------------------------------------------------------
ℹ file | line % | branch % | funcs % | uncovered lines
ℹ ----------------------------------------------------------
ℹ ----------------------------------------------------------
ℹ all files | 100.00 | 100.00 | 100.00 |
ℹ ----------------------------------------------------------
ℹ end of coverage report
Since 23.5.0:
% nvm use 23.5
Now using node v23.5.0 (npm v10.9.2)
% node --experimental-test-coverage --test i.js
23.5.0
✔ foo (0.330042ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 64.134167
ℹ start of coverage report
ℹ -----------------------------------------------------
ℹ file | line % | branch % | funcs % | uncovered lines
ℹ -----------------------------------------------------
ℹ -----------------------------------------------------
ℹ …les | 100.00 | 100.00 | 100.00 |
ℹ -----------------------------------------------------
ℹ end of coverage report
23.4.0 was fine:
% nvm use 23.4
Now using node v23.4.0 (npm v10.9.2)
% node --experimental-test-coverage --test i.js
23.4.0
✔ foo (0.326084ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 62.499375
ℹ start of coverage report
ℹ -------------------------------------------------------
ℹ file | line % | branch % | funcs % | uncovered lines
ℹ -------------------------------------------------------
ℹ i.js | 91.67 | 66.67 | 100.00 | 7
ℹ -------------------------------------------------------
ℹ …files | 91.67 | 66.67 | 100.00 |
ℹ -------------------------------------------------------
ℹ end of coverage report
How often does it reproduce? Is there a required condition?
Every time
What is the expected behavior? Why is that the expected behavior?
Output on 23.4.0 (see above) was fine Non-empty coverage report, reporting line 7 in the above example as uncovered
What do you see instead?
Empty coverage
Additional information
No response
Ah, https://github.com/nodejs/node/pull/56060
But --test-coverage-include i.js doesn't help this, so I was confused
No, this still is confusing
In this setup, i0.js isn't a test file, but is ignored:
i.js:
import { test } from 'node:test'
import { foo } from './i0.js'
console.log(process.versions.node)
test('foo', () => {
foo(2)
})
i0.js:
export function foo(arg) {
if (!arg) {
throw new Error('Failed')
} else {
return arg * 2
}
}
Before 23.5:
23.4.0
✔ foo (0.345917ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 74.277709
ℹ start of coverage report
ℹ --------------------------------------------------------
ℹ file | line % | branch % | funcs % | uncovered lines
ℹ --------------------------------------------------------
ℹ i.js | 100.00 | 100.00 | 100.00 |
ℹ i0.js | 85.71 | 66.67 | 100.00 | 3
ℹ --------------------------------------------------------
ℹ … files | 93.33 | 80.00 | 100.00 |
ℹ --------------------------------------------------------
ℹ end of coverage report
Since 23.5:
23.5.0
✔ foo (6.384833ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 164.013708
ℹ start of coverage report
ℹ -----------------------------------------------------
ℹ file | line % | branch % | funcs % | uncovered lines
ℹ -----------------------------------------------------
ℹ -----------------------------------------------------
ℹ …les | 100.00 | 100.00 | 100.00 |
ℹ -----------------------------------------------------
ℹ end of coverage report
--test-coverage-include=i0.js doesn't help
--test-coverage-exclude='' doesn't work
--test-coverage-exclude='some-invalid-pattern' helps, but is... strange
... ok, this apparently excludes files based on their global location, and if the project is inside a subdir of a .../test/... dir, everything is excluded
which looks confusing, but not very significant to care
Hey @ChALkeR, as per documentation the latest versions are excluding from the coverage the matching test files:
"By default all the matching test files are excluded from the coverage report. Specifying this option will override the default behavior." (https://nodejs.org/docs/latest/api/cli.html#--test-coverage-exclude)
@pmarchini I see that, the issue was that those were not test files
E.g. if an os user is named test, everything in their projects would likely be excluded, as this operates on global paths, not on relative paths
This is unfortunate, but (as I already said) not very significant
Sorry @ChALkeR, I think I didn’t express myself clearly.
You're right about the match pattern.
At the moment, we're excluding, by default, everything that matches the inclusion patterns, which may need clarification.
It might be worth checking whether the documentation makes this clear, since what I shared definitely isn't clear enough
Or perhaps matching should instead be performed on paths relative to the current project root / closest package.json file / cwd as fallback
Or use the files that are matched by node --test
node --test is also documented to include tests matching inclusion patterns, but it does not treat global paths as matches
I would have expected this to exclude the exact same files that are matched by node --test
I think this is a bug.
Hey @targos, IMHO, I think it's working as expected, even though the documentation could be more detailed.
Rather than a bug, I think we should discuss potential alternative solutions!
I would have expected this to exclude the exact same files that are matched by node --test
@ChALkeR if I'm not mistaken we discussed this in another issue (which I can't find at the moment). I agree; that seems reasonable to me.
If I'm not mistaken, the reason we avoided this approach was due to the invasiveness of the required changes. Nonetheless, we could discuss it again.
Not reproducible on Node.js git build.
./node/node --experimental-test-coverage --test i.js
25.0.0-pre
✔ foo (0.3185ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 54.209292
ℹ start of coverage report
ℹ ----------------------------------------------------------
ℹ file | line % | branch % | funcs % | uncovered lines
ℹ ----------------------------------------------------------
ℹ i.js | 90.91 | 66.67 | 100.00 | 7
ℹ ----------------------------------------------------------
ℹ all files | 90.91 | 66.67 | 100.00 |
ℹ ----------------------------------------------------------
ℹ end of coverage report
