Try to understand the output of a damaged JPEG

[shivanraptor]

Hi there, I would like to understand the output of the commands.

% python3 check_mi.py ~/Desktop/error.jpg
********WARNING*******************************************************
You are using Python Pillow PIL module and not the Pillow-SIMD module.
Pillow-SIMD is a 4x faster drop-in replacement of the base PIL module.
Uninstalling Pillow PIL and installing Pillow-SIMD is a good idea.
**********************************************************************
Files integrity check for: /Users/adv/Desktop/error.jpg
Bad file: /Users/adv/Desktop/error.jpg , error detail: [Errno 2] No such file or directory: 'identify' , size[bytes]: 1068217

and for test_damage.py:

% python3 test_damage.py ~/Desktop/error.jpg 
Original image size: 41600
DAMAGE SIZE[bytes] random noise 512
Detected damages 0: 100.0 %
Detected damages 1: 80.0 %
Detected damages 2: 100.0 %
DAMAGE SIZE[bytes] zero-fill portion 512
Detected damages 0: 100.0 %
Detected damages 1: 10.0 %
Detected damages 2: 100.0 %
DAMAGE SIZE[bytes] random noise 16556
Detected damages 0: 100.0 %
Detected damages 1: 90.0 %
Detected damages 2: 100.0 %
DAMAGE SIZE[bytes] zero-fill portion 16556
Detected damages 0: 100.0 %
Detected damages 1: 60.0 %
Detected damages 2: 100.0 %
TRUNCATE SIZE % 1
Detected damages 0: 100.0 %
Detected damages 1: 100.0 %
Detected damages 2: 100.0 %
TRUNCATE SIZE % 10
Detected damages 0: 100.0 %
Detected damages 1: 100.0 %
Detected damages 2: 100.0 %

What does that mean?

[ftarlao]

About the first output, there is an error while checking an image, from the error detail ... the error does not look not about the file being broken BUT the lack of (or installation problem of) the imagemagik libraries, which prevents the tool to work properly. I think the python module --the imagemagik wrapper-- installation should force people/the OS, into imagemagik libs installation. I'll dig into this.

About the second one, test_damage.py is not a script useful for the normal user, it is a suite of mine which I used to get the probability of error detection in relation to the type of damage.

[ftarlao]

Please consider that describing your enviroment is the basic (like 101) when reporting a problem or asking for help. You should be more detailed when submitting a request or issue

[shivanraptor]

Thanks for the reply. I reinstalled the brew formula imagemagick@6 via brew reinstall imagemagick@6 and the problem still persists.

Regarding the environment, here are the details:

• macOS Catalina 10.15.6
• Python 3.9.6
• Using virtualenv 20.4.7
• Imagemagick 6.9.12-17
• Python Pillow 6.2.2
• Python Wand 0.4.5
• Freetype 2.10.4

[ftarlao]

Thank you for the details, I also tried to go deeper into this,

from the terminal/commandline, can you try calling "indentify" command, do you find the command?

The first try is looking for the correct path for the identify command, and add this path to the enviroment path, I have the impression that brew does not force too many search paths into the into the enviroment... for safety reason I think.

The second try: we will comment the identify code in the script (I can also improve by adding a "command available" check, but it comes with imagemagick ...so..)

[ftarlao]

About making improvements I can improve/apply fixes about once every 6-9 months.

[shivanraptor]

Sorry for late reply.

from the terminal/commandline, can you try calling "indentify" command, do you find the command?

no. The Terminal said:

zsh: command not found: identify

[shivanraptor]

After I added the Imagemagick to the path using: echo 'export PATH="/usr/local/opt/imagemagick@6/bin:$PATH"' >> ~/.zshrc, I can use the identify command.

The output of check_mi.py now becomes:

% python3 check_mi.py ~/Desktop/error.jpg 
********WARNING*******************************************************
You are using Python Pillow PIL module and not the Pillow-SIMD module.
Pillow-SIMD is a 4x faster drop-in replacement of the base PIL module.
Uninstalling Pillow PIL and installing Pillow-SIMD is a good idea.
**********************************************************************
Files integrity check for: /Users/adv/Desktop/error.jpg
File /Users/adv/Desktop/error.jpg is OK

However, the JPEG thumbnail is indeed corrupted.

[ftarlao]

Not all image damages are detectable, you can also try strict level 2 (also try to explicitly set strict level 1--the should be the default, but need to double check ), but there are cases where the damage cannot be detected. I'm thinking on an AI based approach wich classify decoded images. But I'll need 6-9 months at least to implement this. Real life .... is hard :-)

Truncation is usually always detected, while zero-fill has a 50% chance of triggering the errors, a random noise in part of the file has an higher ratio. But these limitation are described in the main page.

[shivanraptor]

Thanks for your reply. I tried with strict level 1 & 2. The script still reports OK.

Really thanks for your effort in making this wonderful script.