Benchmarking

[IanButterworth]

I just added a benchmark script at https://github.com/JuliaIO/PNG.jl/blob/master/benchmarking/benchmark.jl

Looks like QuartzImageIO >> PNG > ImageMagick overall, but PNG is fastest to load

All of this is before @drvi's https://github.com/ianshmean/ImageIO.jl/pull/3 which looks like it will improve things 👍🏻

Small image

[ Info: Testing with rand(RGB{N0f8}, 244, 244) image -----------------------------

[ Info: Testing FileIO with ImageMagick (dedicated environment, new instance)
Time to first save (including save): 2.336 seconds
Save @btime:  27.209 ms (3562 allocations: 710.64 KiB)
Load @btime:  5.420 ms (3722 allocations: 717.97 KiB)

[ Info: Testing FileIO with QuartzImageIO (dedicated environment, new instance)
Time to first save (including save): 2.445 seconds
Save @btime:  8.773 ms (1837 allocations: 794.13 KiB)
Load @btime:  3.477 ms (2062 allocations: 574.75 KiB)

[ Info: Testing FileIO with PNG (dedicated environment, new instance)
Time to first save (including save): 1.776 seconds
Save @btime:  20.581 ms (279 allocations: 370.69 KiB)
Load @btime:  2.569 ms (184 allocations: 183.33 KiB)

Large image

[ Info: Testing with rand(RGB{N0f8}, 20000, 4000) image -----------------------------

[ Info: Testing FileIO with ImageMagick (dedicated environment, new instance)
Time to first save (including save): 32.806 seconds
Save @btime:  30.127 s (3562 allocations: 686.83 MiB)
Load @btime:  4.962 s (3729 allocations: 686.84 MiB)

[ Info: Testing FileIO with QuartzImageIO (dedicated environment, new instance)
Time to first save (including save): 14.143 seconds
Save @btime:  11.721 s (1837 allocations: 915.62 MiB)
Load @btime:  3.934 s (2064 allocations: 610.46 MiB)

[ Info: Testing FileIO with PNG (dedicated environment, new instance)
Time to first save (including save): 30.107 seconds
Save @btime:  28.491 s (20035 allocations: 460.82 MiB)
Load @btime:  4.107 s (186 allocations: 228.89 MiB)

[IanButterworth]

As expected, we now win with #2 merged

[ Info: Testing with rand(RGB{N0f8}, 244, 244) image -----------------------------
[ Info: Testing FileIO with ImageMagick (dedicated environment, new instance)
Time to first save (including save): 2.303 seconds
Save @btime:  27.416 ms (3562 allocations: 710.64 KiB)
Load @btime:  5.335 ms (3722 allocations: 717.97 KiB)

[ Info: Testing FileIO with QuartzImageIO (dedicated environment, new instance)
Time to first save (including save): 2.37 seconds
Save @btime:  8.847 ms (1837 allocations: 794.13 KiB)
Load @btime:  3.456 ms (2062 allocations: 574.75 KiB)

[ Info: Testing FileIO with PNGFiles (dedicated environment, new instance)
Time to first save (including save): 2.05 seconds
Save @btime:  4.608 ms (46 allocations: 178.64 KiB)
Load @btime:  3.145 ms (182 allocations: 359.59 KiB)

[ Info: Testing with rand(RGB{N0f8}, 20000, 4000) image -----------------------------
[ Info: Testing FileIO with ImageMagick (dedicated environment, new instance)
Time to first save (including save): 33.791 seconds
Save @btime:  29.370 s (3562 allocations: 686.83 MiB)
Load @btime:  5.350 s (3729 allocations: 686.84 MiB)

[ Info: Testing FileIO with QuartzImageIO (dedicated environment, new instance)
Time to first save (including save): 15.481 seconds
Save @btime:  13.516 s (1837 allocations: 915.62 MiB)
Load @btime:  4.120 s (2064 allocations: 610.46 MiB)

[ Info: Testing FileIO with PNGFiles (dedicated environment, new instance)
Time to first save (including save): 9.059 seconds
Save @btime:  6.418 s (49 allocations: 229.04 MiB)
Load @btime:  4.638 s (185 allocations: 457.92 MiB)

[jaakkor2]

Wondering if ImageMagick and PNGFiles numbers are improving after Zlib_jll got -O3 flag in https://github.com/JuliaPackaging/Yggdrasil/pull/1051.