message-format-wg
message-format-wg copied to clipboard
Create complete tests for syntax
This is from https://github.com/unicode-org/message-format-wg/wiki/Things-That-Need-Doing . I didn't see an existing issue for it.
Full test coverage is probably impossible given the size of the grammar. But there are a couple possibilities:
- Manually generate tests from the grammar, with a limited search depth (i.e. expand every production up to a certain depth and write a test that fits it)
- Use an exhaustive test generator tool like abnfgen, Eusthasius or Gramtest to generate tests.
- Use a fuzzer like abnffuzzer and extract interesting tests to include in the test suite.
As far as option 2, with a few modifications, I was able to run the grammar through abnfgen and Eusthasius. The problem is that both of them generate tests that resemble line noise, suggesting that fuzzing might be preferable.
Option 1 has the advantage that it's easy to write readable, minimal test cases. Of course the disadvantage is that it's tedious.
Any suggestions are welcome.