dangerzone
dangerzone copied to clipboard
Verify PDF output of CI tests
We currently assume that CI tests work because they complete successfully. However, we don't actually validate their output, and we may miss PDFs that have been converted, but are corrupted (see https://github.com/freedomofpress/dangerzone/issues/315).
In order to improve this situation, since the input is always known, we should also include the output in our tests folder, and compare it with that. The problem here is that we can't perform a byte to byte comparison. We need some sort of visual diffing, and ensure that the output PDF is not significantly different from the input PDF.
TIL that this is sometimes called Pixel Tests (see this in PDFium).