Potential for ~2% .. 8% smaller `png`s by more agressive `oxipng` configuration

[CommanderStorm]

This issue is primarily motivated by https://www.openstreetmap.org/user/pnorman/diary/407303

Currently, spreets oxipng configuration is using the default.

https://github.com/flother/spreet/blob/cfc511d0a53fe4bfee0b6a506bc3ea14d9a439f9/src/sprite/mod.rs#L468-L481

In oxipng, this results in -o 2

@pnorman 's blog post uses -o max which equates to -o 6.

I will need to dive deeper into which of the settings affect this the most and see if we can expose this 🤔

[pnorman]

I think the key settings I had were --o max -Z --fast. It took several minutes to optimize the PNG on my 64-core server. Most users will not want this. Users who are planning to serve a spritesheet millions of times can manually run oxipng.

I also had --strip all --alpha but it's likely those had no impact.

[CommanderStorm]

I don't think the default should change. I think it should be configurable though.

This way a user can either

• opt into higher compression in the cli or
• a tile server can do a fast pass and then replace what is in cache with a more compressed variant in the background

I am not sure where exactly the Pareto front lies between effort-outcome.

[flother]

I think Doug McIlroy's Unix philosophy is a useful guiding light here:

Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".

We could add command-line options to configure compression, but I doubt there's much benefit over simply running:

spreet input output && oxipng -o max -Z --fast output.png

What might be better instead is to see if Spreet's defaults could be tweaked. Perhaps there's an Oxipng config setting that would offer better compression without sacrificing speed? Having said that, I assume the Oxipng defaults were chosen for a reason.

[pnorman]

Having said that, I assume the Oxipng defaults were chosen for a reason.

They offer the best size decrease for a reasonable amount of CPU time when applied to general-purpose images.

Sprite sheets are not general-purpose images, so a different set of filters might make sense. Running oxipng -o max -P -vv on an assortment of spritesheets showed the filtersNone, Bigrams, and Brute were the only ones ever found to be optimal or close to it. sprite-one sheets always were none, basemaps/sprites were brute.

spritesheets from spreet had f = None. Could you try some of yours with the command above?

If so, the current settings will be the same as -f 0 --zc 11 since the other filters are never picked. On the osm-carto sheet that takes 1.5s instead of the 4.3 seconds of -o2.

So that's a way to speed things up while getting the same result. Going to --zc 12 doubles the time to 8.2s and decreases size by 4.34%

[flother]

I tried it out on the openstreetmap-americana sprites (249 SVGs producing a 1006x512px spritesheet), using a ratio of 2. As you said, None, Bigrams, and Brute are the filters that have most effect. Raw results below.

Filter	Compression level	User time (secs)	User time change	Bytes	Bytes change
Default Oxipng options	11	0.50	—	137,690	—
None	11	0.42	-16%	137,690	—
Bigrams	11	0.44	-12%	138,551	+0.63%
Brute	11	0.48	-4%	140,044	+1.71%
None	12	0.67	+34%	135,377	-1.68%

From that, it seems like using the default options means no filter (None) is used, but there's a small overhead compared to specifying None explicitly. No filter does better than Bigrams and Brute, both in time and bytes. When I use no filter but increase the compression level to 12, I saw a small reduction in bytes at the cost of a slower run — although .67 secs isn't exactly sluggish.

From that anecdote it seems that Spreet might (slightly) benefit from using no filter rather than the defaults. Moving to level 12 compression doesn't make enough of a change to warrant the slower speed. But I could add something to the README on how to pair Spreet with Oxipng to get that compression.

I'd be interested in trying this out on other sprite sheets, to see if these statements hold true. Does anyone have any public sprites they want to try? It's pretty easy to test for yourself: alter the implementation of encode_png in src/sprite/mod.rs:

https://github.com/flother/spreet/blob/cfc511d0a53fe4bfee0b6a506bc3ea14d9a439f9/src/sprite/mod.rs#L476-L481

To something like this:

// Add imports at the top of the module:
// use oxipng::indexset;
// use oxipng::Deflaters;
pub fn encode_png(&self) -> SpreetResult<Vec<u8>> {
    Ok(optimize_from_memory(
        self.sheet.encode_png()?.as_slice(),
        &oxipng::Options {
            filter: indexset! {oxipng::RowFilter::None},
            deflate: Deflaters::Libdeflater { compression: 12 },
            ..oxipng::Options::default()
        },
    )?)
}

And then test your changes:

cargo build --release
time ./target/release/spreet --retina input output

[pnorman]

I'd be interested in trying this out on other sprite sheets, to see if these statements hold true. Does anyone have any public sprites they want to try? It's pretty easy to test for yourself: alter the implementation of encode_png in src/sprite/mod.rs:

You don't need to do this by modifying code - run oxipng with -Pvv to have it tell you what the smallest filter is

[flother]

Very true, much faster. The code changes are only useful if you want to measure Spreet's speed as well as file size

[pnorman]

The time taken by oxipng within spreet should be basically the same as oxipng as a stand-alone program