image-png icon indicating copy to clipboard operation
image-png copied to clipboard

Don't verify adler32 checksum by default

Open Shnatsel opened this issue 5 years ago • 3 comments

For lightly compressed images adler32 checksum calculation accounts for 50% of zlib decompression time, and the efforts to optimize adler32 implementation itself are largely exhausted.

adler32 allows verification of the final uncompressed bitstream; however, the input file integrity is already verified by crc32 in the PNG format. This makes adler32 verification redundant, since the only things it can detect are:

  1. A bug in the zlib decompression implementation
  2. File corruption so subtle that the outer crc32 missed it

Both of these scenarios are so rare that they're not worth the extra runtime overhead in the default configuration.

Disclaimer: my understanding is based on the summary provided here, I'm not an expert on the PNG format.

Shnatsel avatar Nov 01 '20 20:11 Shnatsel

Even if there weren't an outer checksum, I'd still be quite sympathetic to this idea. File corruption really doesn't seem like much of a concern these days given the reliability guarantees built into lower layers of the stack

fintelia avatar Nov 01 '20 20:11 fintelia

I very much like the idea of a builder-style struct for the decoder that would allow configuring these options, and also to turning that validation off by default. At most it should go into some warning output, if the user provides any.

I also never quite understood what you're supposed to do with a crc failure. Ignore the image? That doesn't seem reasonable, if there's a perfectly fine amount of pixel data. You have a CRC which is supposed to allow you to repair small failures anyways but that's far too expensive on large data blocks—although it would be interesting to offer some optional post processing that at least tries. Is there any of the decoding crates that implements this?

197g avatar Nov 01 '20 23:11 197g

PNG was created in the 90s for network use cases specifically. Back then networks were less reliable and could flip bits at random far more frequently than they do now. I assume you're supposed to re-download the image if CRC doesn't match. But that's just my guess.

From what I've seen, image viewers just ignore CRC mismatches.

Shnatsel avatar Nov 01 '20 23:11 Shnatsel