ImageMagick6
ImageMagick6 copied to clipboard
Published
20 hours ago •
ImageMagick
ImageMagick
different behavior of compare on s390x with 16 bit RGB tiff
Hi,
I'm on Debian unstable with imagemagick 8:6.9.11.24+dfsg-1. Take the following two images as input:
https://gitlab.mister-muffin.de/josch/img2pdf/uploads/b6c682ff74233d4b4e3d42e5a23dd45e/gs-1.tiff https://gitlab.mister-muffin.de/josch/img2pdf/uploads/23ea78a396a7941943ecaf75c9b34a87/normal16.png
md5sums:
d8edb7c0176a827c445d3b27ec905a54 gs-1.tiff
6ad810399058a87d8145d8d9a7734da5 normal16.png
On my amd64 system, the following command succeeds:
compare -metric AE gs-1.tiff normal16.png null:
On s390x though, imagemagick finds 2862 different pixels.
Instead of AE, use RMSE. What is the distortion between the two images? We don't have access to a s390x. Can you issue this command:
identify -verbose gs-1.tiff normal16.png
on both systems and post the results here. The image statistics will give us insight into what the problem could be.
Also post the output from each system for this command:
convert --version
When I use RMSE, then on s390x I get 15779.7 (0.240782). I also don't have a s390x machine around, so I'm using qemu instead. I was made aware of this problem by a Fedora maintainer. On Fedora they are building packages on a range of architectures and some software of mine failed to build on s390x which is how we found this problem. Anyways, if you have a Debian unstable box, you can quickly run s390x code under qemu using mmdebstrap. I have run the same command for amd64 as well as s390x and am posting the output here. First amd64:
$ mmdebstrap --arch=amd64 --include=imagemagick,wget,ca-certificates --customize-hook='chroot "$1" sh -xc "wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/b6c682ff74233d4b4e3d42e5a23dd45e/gs-1.tiff; wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/23ea78a396a7941943ecaf75c9b34a87/normal16.png; convert --version; identify -verbose gs-1.tiff normal16.png; compare -metric AE gs-1.tiff normal16.png null:; compare -metric RMSE gs-1.tiff normal16.png null:"' unstable /dev/null
I: automatically chosen mode: unshare
I: chroot architecture amd64 is equal to the host's architecture
I: automatically chosen format: tar
I: using /tmp/mmdebstrap.deDvg2G6QQ as tempdir
I: running apt-get update...
done
I: downloading packages with apt...
done
I: extracting archives...
done
I: installing packages...
done
I: downloading apt...
done
I: installing apt...
done
I: installing remaining packages inside the chroot...
done
done
done
I: running --customize-hook in shell: sh -c 'chroot "$1" sh -xc "wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/b6c682ff74233d4b4e3d42e5a23dd45e/gs-1.tiff; wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/23ea78a396a7941943ecaf75c9b34a87/normal16.png; convert --version; identify -verbose gs-1.tiff normal16.png; compare -metric AE gs-1.tiff normal16.png null:; compare -metric RMSE gs-1.tiff normal16.png null:"' exec /tmp/mmdebstrap.deDvg2G6QQ
+ wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/b6c682ff74233d4b4e3d42e5a23dd45e/gs-1.tiff
--2020-08-29 06:15:29-- https://gitlab.mister-muffin.de/josch/img2pdf/uploads/b6c682ff74233d4b4e3d42e5a23dd45e/gs-1.tiff
Resolving gitlab.mister-muffin.de (gitlab.mister-muffin.de)... 78.46.39.22
Connecting to gitlab.mister-muffin.de (gitlab.mister-muffin.de)|78.46.39.22|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24494 (24K) [image/tiff]
Saving to: ‘gs-1.tiff’
gs-1.tiff 100%[======================>] 23.92K --.-KB/s in 0.02s
2020-08-29 06:15:29 (1.17 MB/s) - ‘gs-1.tiff’ saved [24494/24494]
+ wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/23ea78a396a7941943ecaf75c9b34a87/normal16.png
--2020-08-29 06:15:29-- https://gitlab.mister-muffin.de/josch/img2pdf/uploads/23ea78a396a7941943ecaf75c9b34a87/normal16.png
Resolving gitlab.mister-muffin.de (gitlab.mister-muffin.de)... 78.46.39.22
Connecting to gitlab.mister-muffin.de (gitlab.mister-muffin.de)|78.46.39.22|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 21728 (21K) [image/png]
Saving to: ‘normal16.png’
normal16.png 100%[======================>] 21.22K --.-KB/s in 0.02s
2020-08-29 06:15:29 (1007 KB/s) - ‘normal16.png’ saved [21728/21728]
+ convert --version
Version: ImageMagick 6.9.11-24 Q16 x86_64 20200718 https://imagemagick.org
Copyright: © 1999-2020 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP(4.5)
Delegates (built-in): bzlib djvu fftw fontconfig freetype heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png tiff webp wmf x xml zlib
+ identify -verbose gs-1.tiff normal16.png
Image:
Filename: gs-1.tiff
Format: TIFF (Tagged Image File Format)
Mime type: image/tiff
Class: DirectClass
Geometry: 60x60+0+0
Resolution: 96x96
Print size: 0.625x0.625
Units: PixelsPerInch
Colorspace: sRGB
Type: TrueColor
Endianness: MSB
Depth: 16-bit
Channel depth:
red: 16-bit
green: 16-bit
blue: 16-bit
Channel statistics:
Pixels: 3600
Red:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.909216
skewness: 0.960573
entropy: 0.445449
Green:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.909216
skewness: 0.960573
entropy: 0.445449
Blue:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.909216
skewness: 0.960573
entropy: 0.445449
Image statistics:
Overall:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.908441
skewness: 0.96084
entropy: 0.445449
Rendering intent: Perceptual
Gamma: 0.454545
Chromaticity:
red primary: (0.64,0.33)
green primary: (0.3,0.6)
blue primary: (0.15,0.06)
white point: (0.3127,0.329)
Background color: white
Border color: srgb(223,223,223)
Matte color: grey74
Transparent color: black
Interlace: None
Intensity: Undefined
Compose: Over
Page geometry: 60x60+0+0
Dispose: Undefined
Iterations: 0
Compression: None
Orientation: TopLeft
Profiles:
Profile-icc: 2576 bytes
Properties:
date:create: 2020-08-29T06:15:29+00:00
date:modify: 2020-06-26T21:36:55+00:00
icc:copyright: Copyright Artifex Software 2011
icc:description: Artifex Software sRGB ICC Profile
signature: 358e596fdd23db91041d1c2083923029654e84036319caefad08b96004566644
tiff:alpha: unspecified
tiff:endian: msb
tiff:photometric: RGB
tiff:rows-per-strip: 22
tiff:software: GPL Ghostscript 9.52
tiff:subfiletype: PAGE
tiff:timestamp: 2020:06:26 21:07:14
Artifacts:
filename: gs-1.tiff
verbose: true
Tainted: False
Filesize: 24494B
Number pixels: 3600
Pixels per second: 10.3973MB
User time: 0.000u
Elapsed time: 0:01.000
Version: ImageMagick 6.9.11-24 Q16 x86_64 20200718 https://imagemagick.org
Image:
Filename: normal16.png
Format: PNG (Portable Network Graphics)
Mime type: image/png
Class: DirectClass
Geometry: 60x60+0+0
Units: Undefined
Colorspace: sRGB
Type: TrueColor
Base type: Undefined
Endianness: Undefined
Depth: 16-bit
Channel depth:
red: 16-bit
green: 16-bit
blue: 16-bit
Channel statistics:
Pixels: 3600
Red:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.909216
skewness: 0.960573
entropy: 0.445449
Green:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.909216
skewness: 0.960573
entropy: 0.445449
Blue:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.909216
skewness: 0.960573
entropy: 0.445449
Image statistics:
Overall:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.908441
skewness: 0.96084
entropy: 0.445449
Rendering intent: Perceptual
Gamma: 0.454545
Chromaticity:
red primary: (0.64,0.33)
green primary: (0.3,0.6)
blue primary: (0.15,0.06)
white point: (0.3127,0.329)
Background color: white
Border color: srgb(223,223,223)
Matte color: grey74
Transparent color: black
Interlace: None
Intensity: Undefined
Compose: Over
Page geometry: 60x60+0+0
Dispose: Undefined
Iterations: 0
Compression: Zip
Orientation: Undefined
Properties:
date:create: 2020-08-29T06:15:29+00:00
date:modify: 2020-06-26T21:33:45+00:00
png:IHDR.bit-depth-orig: 16
png:IHDR.bit_depth: 16
png:IHDR.color-type-orig: 2
png:IHDR.color_type: 2 (Truecolor)
png:IHDR.interlace_method: 0 (Not interlaced)
png:IHDR.width,height: 60, 60
png:sRGB: intent=0 (Perceptual Intent)
signature: 358e596fdd23db91041d1c2083923029654e84036319caefad08b96004566644
Artifacts:
filename: normal16.png
verbose: true
Tainted: False
Filesize: 21728B
Number pixels: 3600
Pixels per second: 14.34MB
User time: 0.000u
Elapsed time: 0:01.000
Version: ImageMagick 6.9.11-24 Q16 x86_64 20200718 https://imagemagick.org
+ compare -metric AE gs-1.tiff normal16.png null:
0+ compare -metric RMSE gs-1.tiff normal16.png null:
0 (0)I: cleaning package lists and apt cache...
done
done
I: creating tarball...
I: done
I: removing tempdir /tmp/mmdebstrap.deDvg2G6QQ...
I: success in 41.6231 seconds
And here s390x:
$ mmdebstrap --arch=s390x --include=imagemagick,wget,ca-certificates --customize-hook='chroot "$1" sh -xc "wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/b6c682ff74233d4b4e3d42e5a23dd45e/gs-1.tiff; wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/23ea78a396a7941943ecaf75c9b34a87/normal16.png; convert --version; identify -verbose gs-1.tiff normal16.png; compare -metric AE gs-1.tiff normal16.png null:; compare -metric RMSE gs-1.tiff normal16.png null:"' unstable /dev/null
I: automatically chosen mode: unshare
I: s390x cannot be executed, falling back to qemu-user
I: automatically chosen format: tar
I: using /tmp/mmdebstrap.L7gxATLPuv as tempdir
I: running apt-get update...
done
I: downloading packages with apt...
done
I: extracting archives...
done
I: installing packages...
done
I: downloading apt...
done
I: installing apt...
done
I: installing remaining packages inside the chroot...
done
done
done
I: running --customize-hook in shell: sh -c 'chroot "$1" sh -xc "wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/b6c682ff74233d4b4e3d42e5a23dd45e/gs-1.tiff; wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/23ea78a396a7941943ecaf75c9b34a87/normal16.png; convert --version; identify -verbose gs-1.tiff normal16.png; compare -metric AE gs-1.tiff normal16.png null:; compare -metric RMSE gs-1.tiff normal16.png null:"' exec /tmp/mmdebstrap.L7gxATLPuv
+ wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/b6c682ff74233d4b4e3d42e5a23dd45e/gs-1.tiff
--2020-08-29 06:11:00-- https://gitlab.mister-muffin.de/josch/img2pdf/uploads/b6c682ff74233d4b4e3d42e5a23dd45e/gs-1.tiff
Resolving gitlab.mister-muffin.de (gitlab.mister-muffin.de)... 78.46.39.22
Connecting to gitlab.mister-muffin.de (gitlab.mister-muffin.de)|78.46.39.22|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24494 (24K) [image/tiff]
Saving to: ‘gs-1.tiff’
gs-1.tiff 100%[======================>] 23.92K --.-KB/s in 0.02s
2020-08-29 06:11:00 (1.03 MB/s) - ‘gs-1.tiff’ saved [24494/24494]
+ wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/23ea78a396a7941943ecaf75c9b34a87/normal16.png
--2020-08-29 06:11:00-- https://gitlab.mister-muffin.de/josch/img2pdf/uploads/23ea78a396a7941943ecaf75c9b34a87/normal16.png
Resolving gitlab.mister-muffin.de (gitlab.mister-muffin.de)... 78.46.39.22
Connecting to gitlab.mister-muffin.de (gitlab.mister-muffin.de)|78.46.39.22|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 21728 (21K) [image/png]
Saving to: ‘normal16.png’
normal16.png 100%[======================>] 21.22K --.-KB/s in 0.01s
2020-08-29 06:11:01 (2.15 MB/s) - ‘normal16.png’ saved [21728/21728]
+ convert --version
Version: ImageMagick 6.9.11-24 Q16 s390x 20200718 https://imagemagick.org
Copyright: © 1999-2020 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP(4.5)
Delegates (built-in): bzlib djvu fftw fontconfig freetype heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png tiff webp wmf x xml zlib
+ identify -verbose gs-1.tiff normal16.png
Image:
Filename: gs-1.tiff
Format: TIFF (Tagged Image File Format)
Mime type: image/tiff
Class: DirectClass
Geometry: 60x60+0+0
Resolution: 96x96
Print size: 0.625x0.625
Units: PixelsPerInch
Colorspace: sRGB
Type: TrueColor
Endianness: MSB
Depth: 16-bit
Channel depth:
red: 16-bit
green: 16-bit
blue: 16-bit
Channel statistics:
Pixels: 3600
Red:
min: 0 (0)
max: 65279 (0.996094)
mean: 18630.3 (0.284279)
standard deviation: 26623.8 (0.406254)
kurtosis: -0.900367
skewness: 0.952804
entropy: 0.445449
Green:
min: 0 (0)
max: 65279 (0.996094)
mean: 18630.3 (0.284279)
standard deviation: 26623.8 (0.406254)
kurtosis: -0.900367
skewness: 0.952804
entropy: 0.445449
Blue:
min: 0 (0)
max: 65279 (0.996094)
mean: 18630.3 (0.284279)
standard deviation: 26623.8 (0.406254)
kurtosis: -0.900367
skewness: 0.952804
entropy: 0.445449
Image statistics:
Overall:
min: 0 (0)
max: 65279 (0.996094)
mean: 18630.3 (0.284279)
standard deviation: 26623.8 (0.406254)
kurtosis: -0.899589
skewness: 0.953069
entropy: 0.445449
Rendering intent: Perceptual
Gamma: 0.454545
Chromaticity:
red primary: (0.64,0.33)
green primary: (0.3,0.6)
blue primary: (0.15,0.06)
white point: (0.3127,0.329)
Background color: white
Border color: srgb(223,223,223)
Matte color: grey74
Transparent color: black
Interlace: None
Intensity: Undefined
Compose: Over
Page geometry: 60x60+0+0
Dispose: Undefined
Iterations: 0
Compression: None
Orientation: TopLeft
Profiles:
Profile-icc: 2576 bytes
Properties:
date:create: 2020-08-29T06:11:00+00:00
date:modify: 2020-06-26T21:36:55+00:00
icc:copyright: Copyright Artifex Software 2011
icc:description: Artifex Software sRGB ICC Profile
signature: 89bc2554fd7d39d029c3fa3554f6641b8d9042a1536750fcdfa49e8b1f764434
tiff:alpha: unspecified
tiff:endian: msb
tiff:photometric: RGB
tiff:rows-per-strip: 22
tiff:software: GPL Ghostscript 9.52
tiff:subfiletype: PAGE
tiff:timestamp: 2020:06:26 21:07:14
Artifacts:
filename: gs-1.tiff
verbose: true
Tainted: False
Filesize: 24494B
Number pixels: 3600
Pixels per second: 176611B
User time: 0.020u
Elapsed time: 0:01.020
Version: ImageMagick 6.9.11-24 Q16 s390x 20200718 https://imagemagick.org
Image:
Filename: normal16.png
Format: PNG (Portable Network Graphics)
Mime type: image/png
Class: DirectClass
Geometry: 60x60+0+0
Units: Undefined
Colorspace: sRGB
Type: TrueColor
Base type: Undefined
Endianness: Undefined
Depth: 16-bit
Channel depth:
red: 16-bit
green: 16-bit
blue: 16-bit
Channel statistics:
Pixels: 3600
Red:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.909216
skewness: 0.960573
entropy: 0.445449
Green:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.909216
skewness: 0.960573
entropy: 0.445449
Blue:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.909216
skewness: 0.960573
entropy: 0.445449
Image statistics:
Overall:
min: 0 (0)
max: 65534 (0.999985)
mean: 18567.9 (0.283328)
standard deviation: 27030 (0.412451)
kurtosis: -0.908441
skewness: 0.96084
entropy: 0.445449
Rendering intent: Perceptual
Gamma: 0.454545
Chromaticity:
red primary: (0.64,0.33)
green primary: (0.3,0.6)
blue primary: (0.15,0.06)
white point: (0.3127,0.329)
Background color: white
Border color: srgb(223,223,223)
Matte color: grey74
Transparent color: black
Interlace: None
Intensity: Undefined
Compose: Over
Page geometry: 60x60+0+0
Dispose: Undefined
Iterations: 0
Compression: Zip
Orientation: Undefined
Properties:
date:create: 2020-08-29T06:11:01+00:00
date:modify: 2020-06-26T21:33:45+00:00
png:IHDR.bit-depth-orig: 16
png:IHDR.bit_depth: 16
png:IHDR.color-type-orig: 2
png:IHDR.color_type: 2 (Truecolor)
png:IHDR.interlace_method: 0 (Not interlaced)
png:IHDR.width,height: 60, 60
png:sRGB: intent=0 (Perceptual Intent)
signature: 358e596fdd23db91041d1c2083923029654e84036319caefad08b96004566644
Artifacts:
filename: normal16.png
verbose: true
Tainted: False
Filesize: 21728B
Number pixels: 3600
Pixels per second: 422584B
User time: 0.010u
Elapsed time: 0:01.008
Version: ImageMagick 6.9.11-24 Q16 s390x 20200718 https://imagemagick.org
+ compare -metric AE gs-1.tiff normal16.png null:
2862+ compare -metric RMSE gs-1.tiff normal16.png null:
15779.7 (0.240782)E: run_chroot failed: E: command failed: chroot "$1" sh -xc "wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/b6c682ff74233d4b4e3d42e5a23dd45e/gs-1.tiff; wget https://gitlab.mister-muffin.de/josch/img2pdf/uploads/23ea78a396a7941943ecaf75c9b34a87/normal16.png; convert --version; identify -verbose gs-1.tiff normal16.png; compare -metric AE gs-1.tiff normal16.png null:; compare -metric RMSE gs-1.tiff normal16.png null:"
W: listening on child socket failed:
I: removing tempdir /tmp/mmdebstrap.L7gxATLPuv...
Let's see if the problem is related to the TIFF delegate library. $ convert gs-1.tiff gs-1.pnm $ compare -metric rmse gs-1.tiff gs-1.pnm null: 0 (0)
Now run `identify -verbose gs-1.pnm` on s390x and post the output. Thanks.
The files gs-1.tiff and gs-1.pnm are indeed identical as expected. Here the output of identify -verbose gs-1.pnm:
Image:
Filename: gs-1.pnm
Format: PPM (Portable pixmap format (color))
Mime type: image/x-portable-pixmap
Class: DirectClass
Geometry: 60x60+0+0
Units: Undefined
Colorspace: sRGB
Type: TrueColor
Base type: Undefined
Endianness: Undefined
Depth: 16-bit
Channel depth:
red: 16-bit
green: 16-bit
blue: 16-bit
Channel statistics:
Pixels: 3600
Red:
min: 0 (0)
max: 65279 (0.996094)
mean: 18630.3 (0.284279)
standard deviation: 26623.8 (0.406254)
kurtosis: -0.900367
skewness: 0.952804
entropy: 0.445449
Green:
min: 0 (0)
max: 65279 (0.996094)
mean: 18630.3 (0.284279)
standard deviation: 26623.8 (0.406254)
kurtosis: -0.900367
skewness: 0.952804
entropy: 0.445449
Blue:
min: 0 (0)
max: 65279 (0.996094)
mean: 18630.3 (0.284279)
standard deviation: 26623.8 (0.406254)
kurtosis: -0.900367
skewness: 0.952804
entropy: 0.445449
Image statistics:
Overall:
min: 0 (0)
max: 65279 (0.996094)
mean: 18630.3 (0.284279)
standard deviation: 26623.8 (0.406254)
kurtosis: -0.899589
skewness: 0.953069
entropy: 0.445449
Rendering intent: Perceptual
Gamma: 0.454545
Chromaticity:
red primary: (0.64,0.33)
green primary: (0.3,0.6)
blue primary: (0.15,0.06)
white point: (0.3127,0.329)
Background color: white
Border color: srgb(223,223,223)
Matte color: grey74
Transparent color: black
Interlace: None
Intensity: Undefined
Compose: Over
Page geometry: 60x60+0+0
Dispose: Undefined
Iterations: 0
Compression: Undefined
Orientation: Undefined
Properties:
date:create: 2020-08-29T11:22:44+00:00
date:modify: 2020-08-29T11:22:44+00:00
signature: 89bc2554fd7d39d029c3fa3554f6641b8d9042a1536750fcdfa49e8b1f764434
Artifacts:
filename: gs-1.pnm
verbose: true
Tainted: False
Filesize: 21615B
Number pixels: 3600
Pixels per second: 348723B
User time: 0.010u
Elapsed time: 0:01.010
Version: ImageMagick 6.9.11-24 Q16 s390x 20200718 https://imagemagick.org
That suggests any bugs are likely in the TIFF delegate library. ImageMagick calls the TIFFReadEncodedStrip() method to read the image pixels. They are unsigned so its not a floating-point issue.