Comparison with mozjpeg?

[magicgoose]

README says:

Guetzli-generated images are typically 20-30% smaller than images of equivalent quality generated by libjpeg

But what about mozjpeg? It existed long before this project, by the way — 1.0 release was in Mar 5, 2014, and it does similar job.

[robryk]

We will soon publish a more detailed comparison that takes mozjpeg and libjpeg into account.

Note that comparing two JPEG encoders is more complicated than it might at first appear, because one needs to compare quality of the compressed image. Obviously, we actually care about quality as perceived by humans, but human evaluation takes time, is costly, and is hard to do correctly, so we often make do with proxies. Traditional metrics (SSIM, PSNR-HVS) are commonly used as proxies; Butteraugli is designed to be such a proxy. These proxies can and do disagree with each other.

[magicgoose]

We will soon publish a more detailed comparison

How soon is soon?

[magicgoose]

By the way, I am almost sure that it if so happens that mozjpeg is more efficient, the comparison won't be published because mozjpeg is Mozilla's project and this is Google™ and Google™ doesn't like to admit failures after scratching their https://en.wikipedia.org/wiki/Not_invented_here syndrome. And this may very well be the reason why there are still no comparisons with mozjpeg in README.

btw, mozjpeg is also ABI-compatible with libjpeg-turbo, this is a valuable thing, and if I understand it right, guetzli isn't that means, it's a lot easier to integrate mozjpeg into existing software

[jyrkialakuijala]

We are still preparing the publication. Other people have run some tests in the mean time. Guetzli is both better and worse than Mozjpeg. https://twitter.com/fg118942/status/820984186974584832 shows an interesting reversal of codecs. Guetzli is the worst on SSIM and PSNR, but best on butteraugli scoring. This is really not a miracle nor a proof of its superiority since guetzli is just a complex optimizer for butteraugli rating.

Ordering of codecs with ssim:

1. BPG
2. JPEG XR/JPEG 2000
3. Mozjpeg
4. JPEG
5. Guetzli

Ordering of codecs with butteraugli:

1. Guetzli
2. BPG
3. Mozjpeg/JPEG
4. JPEG 2000
5. JPEG XR

Note, that butteraugli only looks at the worst area of the image, where as ssim and psnr aggregate errors from everywhere in the image.

I have read that human raters like JPEGs more than similarly sized JPEG 2000 and JPEG XR, but SSIM and PSNR consistently rank them the opposite as humans tend to do.

[acdha]

@magicgoose I'd like to see a rigorous comparison too, but … do you really think the tone of that message is helpful? I don't know what the mood is like at Google but I'm sure it's not helped by unnecessarily politicizing this issue.

[magicgoose]

A little bit of trolling is always good, that's sort of my motto. I'm pleased to see that this was unnecessary and these conspiracy theories are not true. 😄

[MaikWagner]

With some simple convert actions on the console comparing mozjpeg, libjpeg and guetzli with your bees.png sample image, I can't perceive any significant differences in matter of filesize.

convert -quality 85 bees.png bees-libjpeg-85.jpg (using libjpeg) cjpeg -quality 85 bees.png > bees-mozjpeg-85.png.jpg (mozjpeg) guetzli --quality 85 bees.png bees-guetzli-85.png.jpg

-rw-rw-r-- 1 maik maik  20316  6. Mär 13:07 bees-guetzli-85.png.jpg
-rw-rw-r-- 1 maik maik  20128  6. Mär 13:09 bees-libjpeg-85.png.jpg
-rw-rw-r-- 1 maik maik  20359  6. Mär 13:06 bees-mozjpeg-85.png.jpg
-rw-rw-r-- 1 maik maik 177424 24. Okt 17:11 bees.png

Wich comparison method do I have to use to get the 20-30% "typically 20-30% smaller" images with guetzli. Do I have to use it with butteraugli anyway?

[jyrkialakuijala]

Thanks for running these tests on guetzli. This is very helpful.

We have done two different comparisons. One where we keep the filesize the same between guetzli and libjpeg and get humans to look at the pictures, and another where we keep the butteraugli-quality the same and we look at filesize. The first shows a 75-80 % preference to guetzli generated images over the libjpeg images. The latter shows 35 % filesize reduction from libjpeg, and possibly around 30 % from mozjpeg. We didn't do a full human rater study between guetzli and mozjpeg, but a few samples indicated that mozjpeg is closer to libjpeg than guetzli in human viewing.

Our comparisons are done at around qualities 90 to 95. We used a calibrated high quality monitor intended for photo editing work.

Differences at these qualities are subtle.

I think the bees image is one that doesn't suffer a lot from YUV420, and guetzli's aversion to YUV420 might explain why that image works well with convert and mozjpeg. If you try YUV420 with an image where there are red things next to green things, you can observe what is happening in more detail.

[magicgoose]

Not sure why you are pointing at YUV420, mozjpeg can encode full resolution chroma as well.

[robryk]

@magicgoose Note that mozjpeg encodes with YUV420 by default. When you ask mozjpeg to encode with YUV444, the produced image is ~45% larger:

$ ~/mozjpeg/cjpeg -quality 85 bees.png > bees-mj.jpg
$ djpeg -v < bees-mj.jpg 2>&1 >/dev/null | grep -A3 'Start Of Frame'
Start Of Frame 0xc2: width=444, height=258, components=3
    Component 1: 2hx2v q=0
    Component 2: 1hx1v q=1
    Component 3: 1hx1v q=1
$ ~/mozjpeg/cjpeg -sample 1x1 -quality 85 bees.png > bees-mj-444.jpg
$ ls -l bees-mj*
-rw-r----- 1 robryk eng 29705 Mar  6 19:27 bees-mj-444.jpg
-rw-r----- 1 robryk eng 20039 Mar  6 19:21 bees-mj.jpg

[magicgoose]

@robryk I think it's YUV420 by default when quality is less than some number…

also, from my testing, -dct float option in mozjpeg improves efficiency a little bit. -dc-scan-opt 2 may reduce file size a little bit without affecting quality.

[xlz]

http://screenshotcomparison.com/comparison/203655

Some blocky artifacts are more obvious. Just FYI.

[magicgoose]

@xlz can you also upload the source somewhere? I think it'd be interesting to compare it with mozjpeg too (not sure if GIMP uses mozjpeg already)

[jyrkialakuijala]

@xlz Try running guetzli with a higher quality. This shouldn't happen if you use quality 94 or higher.

Guetzli works in a way where image degradation starts to happen everywhere at roughly the same quality settings, it doesn't try to 'save' the easy areas from degradation.

[vinhlh]

I did some tests on my own product images, it seems mozjpeg is better in my case.

Average:

guetzli: reduce 19%.
mozjpeg: reduce 41%.

After reading guetzli papers, it seems my own product images aren't good candidates for guetzli, don't have a lot of pixels can be down-sampled. https://github.com/vinhlh/guetzli-product-image-tests https://vinhlh.github.io/guetzli-tests/public/stats/compare_guetzli.html https://vinhlh.github.io/guetzli-tests/public/stats/compare_mozjpeg.html

[robryk]

@vinhlh Size comparisons make sense between images equivalent in quality. ~~What method did you use to ensure that quality was the same on both sides?~~

I've taken a look at your repository. It seems that you've taken a set of images and did two things with them: a) compressed them with Guetzli, at default quality of 95, b) losslessly reencoded them with Mozjpeg's jpegtran as progressive JPEGs.

Note that (b) doesn't introduce any loss at all. You can do (b) to Guetzlified images too. We've decided not to do anything like (b) in Guetzli, because progressive JPEGs are up to 2x slower to decode.

I've tried re-encoding Guetzlified images in your corpus. The results, compared to yours only-Guetzli and only-reencoding, look as follows (percentages differ slightly from yours, because I've compared the total size change as opposed to averaging the percentages across images):

	size	size reduction
original	16492 kB
Guetzli only	11697 kB	30%
re-encoding only	10030 kB	39%
Guetzli & re-encoding	7490 kB	54%

Thanks for assembling this image corpus and pointing me at it; on most corpora I've seen re-encoding as progressive had smaller gains.

[vinhlh]

@robryk sorry, I didn't know mozjpeg use progressive by default.

~~And I have a question, what you guys meant by~~

Guetzli-generated images are typically 20-30% smaller than images of equivalent quality generated by libjpeg

~~is Guetzli with quality = 100? or with Butteraugli score < 0.97. Could I understand it as if Butteraugli score < 0.97, the compressed and the original image have similar perceived visual quality; and it's very hard for human eyes able to realize the differences.~~ (Same Butteraugli distance)

There's also a Google guy, recommends an optimal Butteraugli score = 1.1. https://medium.com/@duhroach/reducing-jpg-file-size-e5b27df3257c#.a6cx2d5ea

I will do more tests with baseline images and a larger set.

[niutech]

Could you please add JPEGmini to the comparison?

One commenter Ashley Wilson claims his "16MP, 1.71MB file became 1.32MB in JPEGMini, whereas Guetzli only managed to get it to 1.51MB".

[magicgoose]

@niutech who cares? it's a proprietary software, so it's irrelevant.

[acdha]

@magicgoose If JPEGMini gets significantly better results I'm sure a lot of people would like to see an OSS solution become more competitive

[thehelp1]

You all are doing amazing work. Depending on circleci.com you may request certain demands. Within reason!!!

On Mar 27, 2017 2:07 PM, "magicgoose" [email protected] wrote:

@niutech https://github.com/niutech who cares? it's a proprietary software, so it's irrelevant.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/google/guetzli/issues/10#issuecomment-289569877, or mute the thread https://github.com/notifications/unsubscribe-auth/AZNZWIf2EPSDjJhlnT9kq3GNANoVfQGUks5rqBb_gaJpZM4LdX6Q .

[magicgoose]

@acdha yeah that's true, being more competitive is always good. Regardless of whether there exists some non-free software which is claimed to be more powerful.
What bothers me is the claim that some non-free software is better, without any evidence. It looks like a regular marketing job, and IMO it has no place in an issue tracker of a free software project.

[kornelski]

I'd like to remind everyone that any benchmark that only looks at file sizes is essentially invalid. Any such benchmark can simply be won by the worst codec that lowers image quality more than all other tools.

You can say one codec is better than another if it wins on both file size and quality.

But also be careful with "close enough" quality. In JPEG file size grows exponentially with increase in quality, so even differences in quality that are so small that you'd say "looks the same to me", can significantly vary file size, and not because the codec is technically better, but because the settings that were used or re-encoding process allowed it to lose more humanly-imperceptible information.

[kornelski]

In another issue @pflacson posted a link to a direct comparison: https://css-ig.net/articles/guetzli

Note about the comparison: it uses DSSIM metric, but Guetzli deliberately optimizes for its own Butteraugli metric. It definitely succeeds optimizing for its own metric, so in the end it result depends on what measurement you believe is more accurate (and neither of them is perfect).

[jmespadero]

Thanks for bringing here that comparative, Kornel. I have been testing for a week with a large corpus of images, and my conclusions are similar. In my opinion, the better butteraugli distance achieved with guetzli does not pay the incredible amount of resources it needs to encode a file. It is true that with guetzli you can easily beat the filesize of jpeg files created with imagemagick, but if you have in mind to reduce the size of a big set of personal images, you should better read about the parameters of mozjpeg and achieve the same (visual) result in x200 speed.

Also, I must say that this week guetzli has received a set of enhancements that have notably reduced the resources (memory to encode an image went from 300 to 125 ~kBytes~ bytes per pixel), so it is good to keep an eye on it.

[robryk]

@jmespadero Small correction: it's 125 bytes per pixel, not KB per pixel.

[jmespadero]

True. Receive one more time my congratulations for the set of optimizations and enhancements you have done recently.

[vinhlh]

I've tested with a larger set: 500 product images, this is the result:

❯❯❯ ./generate_size_values.sh                                                                                                                                                             
original 79925 kB
guetzli_images 55584 kB, 30% smaller (with guetzli)
jpegtran_images 60094 kB, 24% smaller (with jpegtran -optimize)
moz_images 55325 kB, 30% smaller (with mozjpeg -optimize)
guetzli_prog_images 40178 kB, 49% smaller (with guetzli then mozjpeg -optimize)

[magicgoose]

@vinhlh what about quality metrics?

[vinhlh]

@magicgoose

• guetzli with default quality (95).
• jpegtran/ mozjpeg is lossless, only use -optimize parameter.

Note: tested on a c4.2xlarge instance, it takes 66 minutes for guetzli compression.