qoi
qoi copied to clipboard
phoboslab
Mention that QOI is compressable format
Unlike most of other image formats which can not be compressed further, QOI can be considerably compressed with general compression algorithms Unlike other image formats which will generate highly binary different files for partially same images, QOI generates alike files, alloving to compress variant sets with insane compression (I've got 11MB .qoi.7z out of 4x 12MB pngs)
Variant set: A CG set depicting only a single scenery from a static perspective. (they often have 50-90% of same pixels)
Please write the information about high compression rates for both single images and image sets in the readmes
Yes, I think this point could easily be missed.
In my testing, combining QOI with ZLib compression (QOZ) produces file sizes significantly smaller (roughly 10-25%) than when using PNG format. Of course this gives up most (but not all) of QOI's speed advantages, but QOZ still retains a significant (2-5 times) speed advantage when compressing.
note that zlib is no longer a good choice for compression. QOI+ZSTD or QOI+LZ4 are going to give much better compression vs speed tradeoffs.
I chose ZLib because its libraries are so widely available and for simplicity of adoption.
To me, that feels like a weird argument for an image format that's less than 2 months old.
I agree with @oscardssmith . Limiting recommendations for backwards compatibility when no existing support exists makes no sense
Just a random thought - does re-compressing QOI with QOI yield anything interesting?
no. QOI is very specifically designed for 3 or 4 channel images. The result after QOI compression isn't that.
no. QOI is very specifically designed for 3 or 4 channel images. The result after QOI compression isn't that.
Yeah, sure, I know that. It's just sometimes compression with one algorithm creates certain patterns that can be further compressed with another. bzip2-compressed mysql dumps can be compressed further 10% by gzip, even though the latter is "less sophisticated". This sort of thing.
So for a new compression algorithm it's worth checking if applying it recursively may show some interesting results. It's exceedingly rare, but when it happens it's spectacular.
I did a quick test just now and QOI can't compress further its own output. 400 x 400 photo crop went from 468K to 204K to 269K. So it's a no.
I mean, that's completely unsurprising though. Why would variable-length data chunks of QOI be able to be compressed by opcodes which describe difference-in-RGB movements? lol
I chose ZLib because its libraries are so widely available and for simplicity of adoption.
LZ4 and ZSTD are widely available too, the only likely places that they aren't available is for micro controllers or other hardware that aren't physically capable of running the algorithms. On Debian/Ubuntu adding basic support is the same for all three: Installing liblz4-dev/libzstd-dev/zlib1g-dev from the standard repo, including lz4.h/zstd.h/zlib.h, and the half a dozen lines it takes to call the respective simple encoder/decoder.
Yeah, sure, I know that. It's just sometimes compression with one algorithm creates certain patterns that can be further compressed with another.
A smart QOI-recompressor would at least take advantage of QOI being a byte-aligned format. The tags might be ANS-coded, the RGBA elements may be grouped for further processing, etc. The problem is that QOI output being byte aligned still allows for entropy coders to get good results so a QOI-recompressor would have to be excessively smart to compete.
I chose ZLib because its libraries are so widely available and for simplicity of adoption.
LZ4 and ZSTD are extremely easy to get working in your program. Yesterday I tried using both of them for my own game files package system and it works, just few files *.c and *.h files from their repo, call one function to compress, one to decompress... I recommend trying it out.
dotnet has had a Brotli implementation since core 2.1 (but no zstd) and qoi unsurprisingly works well with Brotli too.
With compression, qoi looks to be better than PNG, perhaps there could be a Compression byte in the header, 0 for none and other values assigned to presently-known algorithms. However, this would significantly complicate interpretation of interchanged .qoi files - i.e. files from diverse sources.
Having gzip or brotili or whatever other compression format wrap a qoi file is good enough; there is no reason to make those external compressions a part of the qoi format.
It can be seen that in many cases, QOI+DEFLATE compresses better than PNG, but sometimes it isn't a better compression. PNG alone is usually better than QOI alone, though. (The below is a list of the contents of the ZIP archive with the test files)
Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------ ------------------------
2021-12-21 04:47:44 D.... 0 0 qoi_test_images
2021-12-21 04:47:44 ..... 349827 333282 qoi_test_images/dice.png
2021-12-21 04:47:40 ..... 519653 350872 qoi_test_images/dice.qoi
2021-11-21 10:18:50 ..... 593463 593449 qoi_test_images/kodim10.png
2021-12-21 04:40:01 ..... 652383 522804 qoi_test_images/kodim10.qoi
2021-11-21 10:19:32 ..... 557596 557596 qoi_test_images/kodim23.png
2021-12-21 04:41:07 ..... 675251 544532 qoi_test_images/kodim23.qoi
2021-12-08 16:06:13 ..... 16605 16526 qoi_test_images/qoi_logo.png
2021-12-21 04:44:14 ..... 16488 8107 qoi_test_images/qoi_logo.qoi
2021-12-21 04:40:17 ..... 14227 13619 qoi_test_images/testcard.png
2021-12-19 15:18:05 ..... 21857 7402 qoi_test_images/testcard.qoi
2021-12-21 04:47:25 ..... 18371 17692 qoi_test_images/testcard_rgba.png
2021-12-19 15:21:21 ..... 24167 9453 qoi_test_images/testcard_rgba.qoi
2016-07-19 08:19:40 ..... 1344960 1328424 qoi_test_images/wikipedia_008.png
2021-12-21 04:42:27 ..... 1521134 1343930 qoi_test_images/wikipedia_008.qoi
------------------- ----- ------------ ------------ ------------------------
6325982 5647688 14 files, 1 folders
Yes, I think this point could easily be missed.
In my testing, combining QOI with ZLib compression (QOZ) produces file sizes significantly smaller (roughly 10-25%) than when using PNG format. Of course this gives up most (but not all) of QOI's speed advantages, but QOZ still retains a significant (2-5 times) speed advantage when compressing.
Yes I confirm. We actually made our internal format based on this: qoi + lzma. You can try it out at https://github.com/sumo-apps/voiconv .. The benefit of "qoi" before lzma is at least to decrease (significally) the size of data going to heavier compression.
Oh my, this thread is full of spam, discussing irelevant off-topic stuff. Including semi-personal attacks, now including mine.
Getting back to the original topic of practical further compression: There are use cases for this, imagine (ha ha) having a game with thousands of images. You want to be able to quickly decompress random files but for downloading that game, you can keep it all relatively small. Or decompress from qoi-set.zstd into memory and decode the qoi lazily.
And then, every dev using qoi as a library can just wrap it in zstd or similar by self, no need to discuss extending the format. There might be even more microcontroller friendly decompressors that might be combined with qoi for tiny flash+memory footprints. Now, benchmarking that kind of stuff would be helpful and bringing this topic forward!
Somebody wants to benchmark a test image set with a selection of some common/uncommon compressors? (Test both single qoi and a set of qoi per compression run!) Then we could add this to the readme!
