unbuild icon indicating copy to clipboard operation
unbuild copied to clipboard

[Suggestion] Add test scaffolding

Open drwpow opened this issue 10 months ago • 0 comments

Summary

This is an optional PR! It does not solve an existing issue but may make contribution easier. Please review and accept or reject at your discretion.

Reasoning

In investigating adding support for #448, I noticed there weren’t many build tests. I didn’t have an easy way to extend the existing tests to first see if a certain config generated a certain output. So I started expanding the tests so that one could easily tweak a single build option and test its output.

This snapshots files which IMO is the best solution for build tools like this for the following reasons:

  • Build tools like unbuild MUST make certain guarantees to purity. Meaning, as long as the inputs don’t change, the output shouldn’t either. Thus, snapshots shouldn’t flake, in theory (given tightly-scoped inputs, which these tests have)
  • When it comes to tests that test complex build systems like this, manual assertions are difficult to impossible to write. Think about all the config options, all the files generated, every little nuance of everything (how would we test that all sourcemap comments in all files are correct?)—there are so many scenarios that are easy to miss. But with snapshots, new tests get 100% coverage automatically with no work.
  • Because any potential output change may be a breaking change, snapshots guard against accidental bugs. Erring on any change in output is desirable because it is likely a regression.
  • The downside to snapshots is they only test changes, not desired behavior and for that reason are not suitable for all scenarios. But I would like to argue that they are fantastic for build systems where every change MAY be a regression and surfacing any change as a potential error for inspection is desirable

Reviewing

Mostly would like feedback on:

  • Would this setup improve coverage for existing build options?
  • Would this test setup be easy to expand on (especially for contributors like myself)?
  • Anything additional that we’d want to test?

drwpow avatar Feb 08 '25 00:02 drwpow