Coverage with c8 and tsx reporting as 100% since version 4.3.0

[OscarBarrett]

Acknowledgements

• [X] I searched existing issues before opening this one to avoid duplicates
• [X] I understand this is not a place for seek help, but to report a bug
• [X] I understand that the bug must be proven first with a minimal reproduction
• [X] I will be polite, respectful, and considerate of people's time and effort

Minimal reproduction URL

https://github.com/OscarBarrett/tsx-coverage-issue

Version

v4.6.2

Node.js version

v20.10.0

Package manager

yarn

Operating system

Linux

Problem & Expected behavior

We use c8 and tsx together when running our test suites, and since tsx version 4.3.0 any file that is imported in our tests gets 100% coverage. Locking to tsx 4.2.1, we get the expected coverage.

Assuming this is related to #405

Contributions

• [ ] I plan to open a pull request for this issue
• [ ] I plan to make a financial contribution to this project

[andreashuber69]

I have the same issue. FWIW, you can work around this by running tests on the built js output. If you have TypeScript generate .map files, then the CC output should look the same as before (i.e. reported on ts files rather than js files). Here's an example of the changes necessary (besides building with tsc):

https://github.com/andreashuber69/verify-coldcard-dice-seed/commit/35b72b9bcc4a087790848db0afb3df9407b0d919

[jbergstroem]

Ran into the same; just wanted to chip in and confirm that downgrading works.

[privatenumber]

I haven't investigated this but I'm thinking this is as simple as removing this check so sourcemaps are always inlined: https://github.com/privatenumber/tsx/blob/8c778919496a8164ec63d0b488c805bce1aba839/src/source-map.ts#L23

There will need to be tests though. PR welcome if anyone wants to tackle.

[cenfun]

This issue is related to the source maps conversion with v8-to-istanbul, and it is not related to tsx. see here V8 provides the correct coverage data

[
  [ { startOffset: 0, endOffset: 592, count: 1 } ],
  [ { startOffset: 47, endOffset: 113, count: 2 } ],
  [ { startOffset: 114, endOffset: 143, count: 1 } ],
  [ { startOffset: 161, endOffset: 195, count: 0 } ]  <- uncovered 
]

However, it has not been applied to the source file index.ts

[
  CovLine { line: 1, startCol: 0, endCol: 0, count: 1, ignore: false }
]

I built a tool that can be used to generate native v8 coverage reports, you might have a try

npm i monocart-coverage-reports -g
mcr npx tsx bin/test.ts -r v8,console-details --entryFilter **/*.ts

┌─────────────────┬──────────┬────────────┬──────────┬───────────┬──────────┬─────────────────┐
│ Name            │    Bytes │ Statements │ Branches │ Functions │    Lines │ Uncovered Lines │
├─────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ bin             │          │            │          │           │          │                 │
│ └ test.ts       │ 100.00 % │   100.00 % │          │           │ 100.00 % │                 │
│ tests           │          │            │          │           │          │                 │
│ └ maths.spec.ts │ 100.00 % │   100.00 % │ 100.00 % │  100.00 % │ 100.00 % │                 │
│ index.ts        │  55.47 % │    50.00 % │ 100.00 % │   50.00 % │  50.00 % │ 5-7             │
├─────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ Summary         │  89.90 % │    75.00 % │ 100.00 % │   66.67 % │  84.21 % │                 │
└─────────────────┴──────────┴────────────┴──────────┴───────────┴──────────┴─────────────────┘

example: https://github.com/cenfun/mcr-tsx

[privatenumber]

Sorry, I'm going to hide your comment because it doesn't lead to a solution in tsx.

Seems like both versions of tsx in the reproduction doesn't work in the latest version of Node, but the reproduction still works fine with Node v20.10.0.

I'm curious what happened between tsx v4.2.1 and v4.6.2 that broke c8 in Node v20.10.0.

[cenfun]

@privatenumber the generated sourcemap content could be different could be caused by this

/**
	 * Improve performance by generating smaller source maps
	 * that doesn't include the original source code
	 *
	 * https://esbuild.github.io/api/#sources-content
	 */
	sourcesContent: false,

[privatenumber]

@cenfun

You're right!

I tested removing sourcesContent (https://github.com/privatenumber/tsx/compare/esbuild-sourcesContent?expand=1) in the reproduction and the coverage is working again in Node 20.10.0 and v20.12.1 (latest LTS):

pnpm i 'privatenumber/tsx#npm/esbuild-sourcesContent'

(Here's how to make a installable branch, if you're interested)

I'll need to look into a way to add tests for this.

[privatenumber]

This has been completed in the development repo.

[privatenumber]

:tada: This issue has been resolved in v4.10.4

If you appreciate this project, please consider supporting this project by sponsoring :heart: :pray:

[wellwelwel]

🎉 This issue has been resolved in v4.10.4

An observation that I find interesting to relate is that after v4.10.4, the coverage of a project dropped by almost 2%. Looking at the differences between tsx versions, I noticed that it was mainly the imports of types.

To fix it, I just used c8's recommendation:

/* c8 ignore next */
import type { SomeType } from '../some-type.js';

• I don't see this as an issue, so I preferred to add it to the context of this issue.

@privatenumber, thanks for this project and the fix 💙

[rozzilla]

Hi @privatenumber 👋🏼 😊 I also have a similar issue. I opened it here.