Integer overflow in psd.c causing undefined behaviour?

[agoodm88]

ImageMagick version

6 (head)

Operating system

Linux

Operating system, version and so on

Ubuntu 20.04

Description

I've been trying to fuzz the PSB/PSD functionality in ImageMagick 6 recently. On each run I eventually ran into poor stability score.

I believe I've narrowed the cause down to integer overflows affecting the following code lines:

349 SetPixelRed(q,(GetPixelRed(q)-((1.0-gamma)*QuantumRange))/gamma); 350 SetPixelGreen(q,(GetPixelGreen(q)-((1.0-gamma)*QuantumRange))/gamma); 351 SetPixelBlue(q,(GetPixelBlue(q)-((1.0-gamma)*QuantumRange))/gamma);

417 SetPixelAlpha(q,(Quantum) (QuantumScale*(GetPixelAlpha(q)*opacity)));

1363 layer_info->channel_info[channel].size-2,SEEK_CUR);

Unfortunately my understanding over what is going on here is limited as this is my first time dabbling in this area. My main purpose for completing this was to soak test some new hardware; then I appear to have fallen into a rabbit hole!

Impact appears to be potential memory disclosure as for some testcases if you output to lossless format and then convert the resulting output back to PAM there appears to be fragments of the input and / or memory like structures visible.

Reproducers attached.

Steps to Reproduce

Imagemagick6head compiled with Ubsan.

convert PSB:reproducer PAM:/dev/null

Images

undefined_exploration.zip

[urban-warrior]

Unfortunately we cannot reproduce the problem. We're using ImageMagick 6.9.12-29 built with:

./configure CFLAGS="-g -fsanitize=address -fno-omit-frame-pointer" CXXFLAGS="-g -fsanitize=address -fno-omit-frame-pointer" LDFLAGS="-g -fsanitize=address -fno-omit-frame-pointer" --disable-shared && make -j; make -j install
convert 'id:001356,sync:fuzzer12,src:001345,+cov' /dev/null

We tried for each of your test images and the commands completed without complaint.

We have received a number of potential issues that only seem to occur under Ubuntu suggesting a compiler chain or libgomp issue. In most cases we cannot reproduce under our Fedora 35, CentOS 8, and Windows 10 development hosts.

[agoodm88]

You have forgotten to compile with ubsan. Here is my configure line:

CC=/usr/bin/afl-clang-fast CXX=/usr/bin/afl-clang-fast++ CFLAGS="-ggdb -fsanitize=address,undefined -fno-omit-frame-pointer" LDFLAGS="-ggdb -fsanitize=address,undefined -fno-omit-frame-pointer" ./configure --disable-shared --without-threads --without-magick-plus-plus --without-bzlib --without-zlib --without-zstd --without-dps --without-flif --without-fpx --without-djvu --without-fontconfig --without-freetype --without-raqm --without-gdi32 --without-heic --without-jbig --without-jpeg --without-jxl --without-lcms --without-openjp2 --without-lqr --without-lzma --without-openexr --without-pango --without-png --without-raw --without-tiff --without-webp --without-xml

[urban-warrior]

Thanks for the problem report. We can reproduce it and will have a patch to fix it in the GIT main branch @ https://github.com/ImageMagick/ImageMagick later today. The patch will be available in the beta releases of ImageMagick @ https://imagemagick.org/download/beta/ by sometime tomorrow.

[agoodm88]

Thanks for the update. I notice a quite considerable amount of changes here so I am impressed you were so fast :-) Are you able to provide any detail regarding the issue, potential impact, etc?

[urban-warrior]

There were other undefined behaviors due to to an overflow condition that we fixed along with the one you identified. We will continue to determine if UBsan identifies additional issues and patch as needed. Once we complete the work, we will issue a new release with the patches.

[agoodm88]

Please could you provide an update regarding your work?

[urban-warrior]

Problem fix. Source/binaries released. Patches required to mitigate the issue is posted above for IM7 and IM6.