Integration Module with ApprovalTests.Java

[jlink]

Testing Problem

Approval testing is an interesting approach, especially when dealing with legacy code. Combining it with (deterministic) test generation as done by jqwik could give it real additional value. ApprovalTests.Java is the framework of choice for doing AT in Java. It sort of works with jqwik, but not in a very convenient way.

Suggested Solution

Create a jqwik module which builds on the ideas in this issue for ApprovalTests.Java

Discussion

There may be changes warranted in ApprovalTests.Java to simplify the implementation of a jqwik-approvaltests module. Maybe there could be native support for jqwik contributed as a PR. The different options should be researched in a POC first before deciding on the best way to go.

[jlink]

Could be interesting for you https://github.com/mmerdes.

[adam-waldenberg]

So essentially, all this is is, is an Approvals module/class that output what it receives into a file... This can then be renamed to approve the received output on the next run.

This is usually done manually in JUnit tests when you expect a certain output - I can see how this could be useful. Question is though, can it be integrated with property tests? From where I am looking it mainly seems useful with @Example tests only.

[jlink]

This is usually done manually in JUnit tests when you expect a certain output - I can see how this could be useful. Question is though, can it be integrated with property tests? From where I am looking it mainly seems useful with @Example tests only.

I tend to think so, because it could simplify and empower the Approvals.verifyAll(..) and Approvals.verifyAllCombinations(..) cases. See the example in the referenced issue.

The standard @Example is already working, since this works more exactly as the standard Jupiter test method.

[adam-waldenberg]

Yeah ... It could be pretty useful - it won't work with that library though from what I can tell? There doesn't seem to be any way to save any userdata on invocations (so you can keep track of things like invocation # and seed #). I see there is something for parameterized tests in there (although, the userdata in this case is just a string and it outputs additional files for each parameterized version... which isn't very useful when you have 1000's of invocations): https://github.com/approvals/ApprovalTests.Java/blob/master/approvaltests/docs/how_to/ParameterizedTest.md#to

Rather than modifying that library, a native solution with an Approvals module combined with a reflection library that gathers "deep" versions of objects seems simpler.

Sorry if I'm rambling... I just started thinking out loud... ;)