test_runner: add inline snapshot assertion

[rluvaton]

TODO

• [x] Add tests
• [x] Add documentation

[nodejs-github-bot]

Review requested:

• [ ] @nodejs/test_runner

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/61326/

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 87.34%. Comparing base (cc26951) to head (3c4209d). Report is 592 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #54480   +/-   ##
=======================================
  Coverage   87.34%   87.34%           
=======================================
  Files         649      649           
  Lines      182544   182541    -3     
  Branches    35030    35033    +3     
=======================================
+ Hits       159445   159449    +4     
+ Misses      16372    16364    -8     
- Partials     6727     6728    +1     

Files with missing lines	Coverage Δ
lib/internal/test_runner/snapshot.js	100.00% <100.00%> (ø)
lib/internal/test_runner/test.js	98.36% <100.00%> (+<0.01%)	:arrow_up:

... and 20 files with indirect coverage changes

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/61332/

[cjihrig]

I'm curious how valuable this type of assertion is given the fact that there is no update functionality. My understanding is that runners like Jest will rewrite the source code with the snapshot value. Without that aspect, I'm not sure what the advantage is over a regular assertion.

Somewhat related to this - I think it would be useful to expose an API for adding (and possibly removing) functions to t.assert. Snapshot testing was an internal proof of concept for that. I plan to continue work on that API after I'm finished with my planned changes for Node v23, but if someone else wants to do it first, that would be great. The changes in this PR to lib/internal/test_runner/test.js show how relatively simple it could be.

[rluvaton]

I'm curious how valuable this type of assertion is given the fact that there is no update functionality. My understanding is that runners like Jest will rewrite the source code with the snapshot value. Without that aspect, I'm not sure what the advantage is over a regular assertion.

In my opinion it would be very useful, my use case is snapshot testing some error output that printed to console and I have default serializer to convert the ANSI codes for example jest-serializer-ansi-escapes

Somewhat related to this - I think it would be useful to expose an API for adding (and possibly removing) functions to t.assert. Snapshot testing was an internal proof of concept for that. I plan to continue work on that API after I'm finished with my planned changes for Node v23, but if someone else wants to do it first, that would be great. The changes in this PR to lib/internal/test_runner/test.js show how relatively simple it could be.

I agree, although personally, I prefer the expect syntax so I mostly use this except for snapshot testing, in which I use the node:assert package

I want to add property matchers to snapshots like in jest to support dynamic data which is extremely useful

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/61338/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/61340/

[cjihrig]

I agree, although personally, I prefer the expect syntax so I mostly use this except for snapshot testing, in which I use the node:assert package

Yea, that's the goal of making t.assert customizable by users. You can already use chai or expect for assertions. This would make them usable on test contexts as well. Also, snapshots are not part of node:assert 😄

[rluvaton]

@cjihrig are you blocking on this?

[benjamingr]

He's not (afaict) but he's not convinced that this feature as-is is useful enough and neither am I (given it's much worse dx than the userland alternatives that rewrite your code)

[rluvaton]

Neither am I (given it's much worse dx than the userland alternatives that rewrite your code)

Do you mean in jest?

Using inline snapshot we can use string manipulation on the output which we can't do in the regular snapshot, like creating dynamic snapshot text that depends on the process ID that runs it or (in my use case) ignoring the random port that the server picked (when using port 0)

[cjihrig]

I'm not blocking, I just foresee people opening issues over the lack of update functionality, even when --test-update-snapshots is passed. Without update functionality, this is essentially only assert.strictEqual(serializers(actual), expected).

I want to add property matchers to snapshots

I realized I never commented on this bit. I think that's a useful feature. I'm curious what your API design would look like since it is based on expect in Jest.

[github-actions[bot]]

This issue/PR was marked as stalled, it will be automatically closed in 30 days. If it should remain open, please leave a comment explaining why it should remain open.

[github-actions[bot]]

Closing this because it has stalled. Feel free to reopen if this issue/PR is still relevant, or to ping the collaborator who labelled it stalled if you have any questions.