libheif
libheif copied to clipboard
initial version of regression test framework
This is a concept draft for regression testing. It was inspired by the iden
regression going unnoticed.
We do have unit testing, but it doesn't really exercise a lot of the functionality.
So this makes use of pytest (which is assumed installed - its a package on ubuntu) to do black-box testing. The main check at this stage is that heif-info produces the right result. There are a bunch of other things that could be tested.
Key decisions are:
- Is this worth the effort to maintain?
- Are the tests reliable? (I'm not sure what I have is reliable).
- Is there an additional / better source of test data?
- Do we want to do image comparisons against known-good snapshots (e.g. PNG files)
A couple of other things I noticed:
- The output of
heif-info --dump-boxes
puts the MIME type, main brand and compatible brand at the start when you are using it interactively, and at the end when redirected to a file (e.g. see the last three lines in any of the .dump files). - The output of
--help
goes to stderr. That is probably OK for a mis-use, but might not be appropriate when requested, since its not really an error at this point.
Thoughts / opinions welcome.