smlfmt icon indicating copy to clipboard operation
smlfmt copied to clipboard

Autoformatter tests

Open HarrisonGrodin opened this issue 4 years ago • 3 comments

It seems like it'd be a good idea to have a test suite for the autoformatter. At a base level, might be good to write "expectation tests", i.e. pairs of files and their autoformatted counterparts, so git diffs would at least show how style changes.

Given expectation tests, might be good to test some desired properties:

  • Idempotence: format o format = format. Can run the formatter twice and make sure the output is the same as formatting once.
  • Token-preserving: can check that lex file = lex (format file)
  • Comment-preserving: can insert comments immediately before and after each token and make sure they stay (including relative position to tokens).

HarrisonGrodin avatar Dec 25 '21 23:12 HarrisonGrodin

We could also develop a linter, which doesn't modify code but rather just outputs suggestions for better style. Then we could use the linter to check the output of the formatter.

The linter would of course be separately useful on its own.

An example of something the linter could check is: make sure all declarations in a sequence appear at the same indentation level. Similarly, case branches, function clauses, etc. (Some exceptions here for flattened code, e.g. let ... in ... end all in one line.)

shwestrick avatar Jan 04 '22 18:01 shwestrick

Yes, this would be extremely useful! Definitely want to work on this. (Hopefully, the linter could recognize type-driven issues, too.)

HarrisonGrodin avatar Jan 04 '22 21:01 HarrisonGrodin

haha yes one of these days we'll have to implement the type system!

shwestrick avatar Jan 04 '22 22:01 shwestrick

As of #77, we have a flag --check which checks for token-preservation, comment-preservation, and idempotence. (I've found a few edge cases where idempotence is not preserved, so one of the next things will be to fix that.)

shwestrick avatar Jan 18 '23 05:01 shwestrick