Fix memory allocation issue with `identify`.

[firmart]

Use ulimit to bound identify in

• space: 100MB + filesize
• time: 3 seconds

ISSUE TYPE

• Bug fix

RUNTIME ENVIRONMENT

• Operating system and version: Ubuntu 19.04 x86_64
• Terminal emulator and version: XTerm(330) and termit 3.0.0
• ranger version: ranger 1.9.2
• Python version: 3.7.3 (default, Oct 7 2019, 12:56:13) [GCC 8.3.0]
• Locale: en_US.UTF-8

CHECKLIST

• [X] The CONTRIBUTING document has been read [REQUIRED]
• [X] All changes follow the code style [REQUIRED]
• [X] All new and existing tests pass [REQUIRED]
• [ ] Changes require config files to be updated
  • [ ] Config files have been updated
• [ ] Changes require documentation to be updated
  • [ ] Documentation has been updated
• [ ] Changes require tests to be updated
  • [ ] Tests have been updated

DESCRIPTION

There is an excessive memory allocation issue when ranger is using identify in scope.sh
to query the orientation of the file of mimetype image/*. (See #1815 #1929 #1946).

I suggest to bound identify resources both in time and space (100MB + filesize and 3 seconds) and recover this limitation later. Moreover, since the mimetype of djvu is categorized under image, I suggest to exclude it and only match jpeg and png.

MOTIVATION AND CONTEXT

Bug: #1815 #1929 #1946

TESTING

• Python code are not modified (all tests passed).
• .png, .jpeg, .pdf are still correctly displayed. .djvu is excluded.

[firmart]

By "all tests passed" I mean those of pytest. make test which uses pylint gives me runtime error even If I didn't change a python loc.

[notaLonelyDay]

Your changes don't work on my machine. It doesn't preview images. Now I see raw text instead of preview

[firmart]

@ThePonyCoder What's the mime-type of the images you test on? My change excludes those other than image/png and image/jpeg.

[notaLonelyDay]

@firmart yea, I've tried jpeg, jpg, png and others. I don't know, maybe something wrong in nixos

[toonn]

Restricting the mime types is not an option imo. Djvu can be used perfectly well to store images. That also happens to be its main use, albeit in the form of scanned documents, hence the mime type.

But there's tons of other image formats people want to preview, raw for example or webp, but even common ones like gif and tiff that you're excluding.

Be sure to use pylint 2 if you want to run the linting.

[firmart]

You're right, excluding is in fact not necessary because the process dies naturally if it exceeds the limit. However, it will respawn if the file is still selected. The ulimit should fix #1929 anyway. I'll continue trying tomorrow with djvu containing images and without.

[firmart]

@toonn I ended up with a solution which lets me preview djvu files (for the first time!). The first solution is abolished, but I messed up when I tried to squash them. Consequently, this doesn't fix #1929 anymore, but I guess it may be the expected behaviour of identify when it deals with 30 MB png.

[glyh]

Hope this will be merged. On my computer ranger call so many identify instances that eat up all of my memory. (In a directory with 99 files, and most of which aren't images)

[toonn]

Wow, almost 2 years : s This one definitely slipped through the cracks. I'll be testing it later today and merging.

@glyh, could you report on #1929 instead of here? In either case this PR won't help you unless the source of the problem is DjVu files.

EDIT: Couldn't get around to it today, will try tomorrow.

[elliotwesoff]

I'm getting this issue with djvu files. Is this going to be merged, or can I fix it locally somehow?

[toonn]

I forgot about this and resigned myself to the same fix, basically, in #2743. Now I'm thinking maybe we need to allow fall-through to the general image case again because DjVu is a valid image format. Testing of both version of the solution is welcome though. I don't know why this one relies on convert rather than simply output a TIFF from ddjvu for example. Maybe some image preview methods aren't compatible with TIFFs?