Improving image preview performance

[Microwave-WYB]

trafficstars

Runtime Environment

• Operating system and version: Manjaro Linux
• Terminal emulator and version: kitty 0.26.3
• Python version: 3.10.7
• Ranger version/commit: ranger-master
• Locale: en_US.UTF-8

Current Behavior

Image preview is slowing down file navigation. You have to wait until the preview is loaded before moving to the next file

Expected Behavior

Image preview should not slow down file navigation

Context

When browsing a lot of image files, especially when these files are large, it is very hard to navigate smoothly between files because you are always waiting for this huge preview to be loaded.

Possible Solutions

1. Interrupt image preview with file navigation
2. Generate smaller preview for large files to improve performance
3. Cache more image previews in the directory

Steps to reproduce

1. Open a folder containing large images
2. Navigate between files with j and k

[Microwave-WYB]

I actually tried to implement a multi threaded image preview so that preview will not stop you from moving, but I'm encountering weird bugs when too many threads are spawned in a short time. Another possible thing we can do is to preprocess down-scaled image thumbnails in the background and store in a cache, instead of down-scaling the image each time the user is trying to access the image.

[Friptick]

Any comments on this, @toonn ? It seems a clone of #1808, which you closed as unfixable.

I am using the Kitty implementation and, as @PoisonFrog says:

Unfortunately it is just better to use ueberzug (ueberzugpp) as the image preview method. The kitty implementation in ranger is slow and crashes when you use commands.

The ueberzug method leaves ghost windows everywhere and is basically unusable, but it is much faster.

I also use feh and that is also much faster at going thru a directory, even in full-screen mode. It does not seem to be caching the images (or at least I have not located the cache).

On a related issue, you pointed out that:

Another factor that'll slow down things like network filesystems is ranger's tendency to do a lot of stat calls.

Additional anecdata: I have a feh plugin script that loads file info for each image (an ls call, so like stat) and that slows down feh significantly.

Absolutely not an expert, but does this not all suggest that a major part of the sluggishness might come from those calls? I guess they are being used to populate the status bar, which suggests a quick fix - a setting to disable the status bar! In my personal workflow I hardly every look at the status bar anyway, would be annoying if that is what is slowing things down.

Thoughts?

NB. Thanks for all your work.

[toonn]

@Friptick, no, if the stat calls were to blame then the Ueberzug method wouldn't be fast either.

The problem comes down to Ranger being fully synchronous, it can't do more than one thing at a time, so when it's generating a preview using PIL it can't respond to input.

The obvious solution is to make Ranger asynchronous, but that's an enormous task. And it brings new problems with it, what Microwave-WYB ran into for example, you can't just go on a thread-spawning rampage.

So you need to cancel some of the preview generation requests. Which ones should we cancel first? Well, the oldest requests are probably for files that we scrolled past longer ago, so in a sense they're far away and we're unlikely to go all the way back, while we might go down a bunch and then two back up, because we went past what we were looking for or we decide we feel more like watching that previous movie anyway.

Problem solved, cancel whatever's the oldest request first. Well hold on, that oldest request has probably been running for the longest, so it's closest to being done and once a preview is generated Ranger caches it and next time it will be much faster to display. That means cancelling the oldest request first wastes the most processing power. The most recent request has actually barely managed to do anything so cancelling it is almost free.

Compared to the real difficulties with asynchronous systems this is really a trivial issue but I hope it illustrates how this is actually a hard problem to solve.

[Friptick]

Thanks for the explanation @toonn

So in the end I could bear this no more and moved to one of the Ranger reimplementations (which one should be unimportant). And rolled my own very simple handler for the image preview, which caches them on first load in much the way you describe. It's sluggish on first load, as expected and as you describe. But very fast subsequently, also as expected - but not what I experienced with Ranger. Thoughts?

I'm wondering whether the caching was somehow broken in my Ranger installation. I did do some digging to check but had some trouble understanding what was going on. I know that default settings were not changed.

PS As for the synchronicity issue, my cache-builder script just bails out on any unfinished previews if I cycle too fast thru new images. Suboptimal but not unexpected behavior, certainly better than being slow every single time. But maybe my Ranger caching was just broken as mentioned.

[toonn]

@Friptick, it sounds like whatever you switched to also does synchronous calls to your cache-builder script, otherwise I'm not sure how you're getting the signal to bail out?

If that is the case then Ranger shouldn't really be any slower, there can be a factor of performance difference due to all the various transformations, like scaling or rotating, that scope.sh tries to apply for example but it shouldn't be equally slow every time you try to preview the exact same file.

It would be interesting to figure out what exactly was going wrong in your case but I honestly don't have a good way of finding out.

[Friptick]

Thanks. Yes, if nobody else has complained then it look like I might have had something odd that was breaking the caching. Go ahead and close. Thanks again for your work on this otherwise excellent project.

[toonn]

OK, let me know if you figure out why it was happening.