The on-demand thumbnails downloader makes everything super slow when failing

[bslenul]

Is there an existing issue for this?

• [X] This is a bug in RetroArch frontend
• [X] I have searched the existing issues

Description

At the time I'm opening this issue https://thumbnails.libretro.com is down, and I noticed it makes loading games much slower but also some other actions like loading a state or just closing RetroArch.

Here's an example:

https://github.com/user-attachments/assets/52752d95-4ff0-456f-919f-4f7c9e9cbfcc

1. Existing thumbnails don't load.
2. Hangs for a second or 2 when opening the game.
3. Doesn't load state until the download failed.
4. Not shown in the video but RetroArch hangs for a few seconds on close if I go to my "History" tab then close RA (I'm assuming it's waiting for the download to fail).

edit: Here for 4.:

https://github.com/user-attachments/assets/70c269cd-a200-4b59-9966-0c7fd169f50c

I use the log console to show that it takes waaaaaaay longer to close than usual (it should close almost instantly) but you can also use the task manager or whatever to see that retroarch.exe stays open for a few seconds after closing.

Note that it seems to happen ONLY if you have a game with missing thumbnail in your "History" tab.

Expected behavior

No response

Steps to reproduce the bug

Might be hard to test if the server goes back ON, I guess you could block the address in firewall or something? Idk. But anyway:

1. Enable Settings > Playlists > On-Demand Thumbnail Downloads.
2. Restart RetroArch.
3. Go to "History" tab with at least one game with missing thumbnails.
4. Launch a game (and optionally load a state) or just close RetroArch.

Version/Commit

1.19.1 29bee5c

Bisect Results

No response

Check in the nightly version

Yes, this is reproduced in the nightly build

Platform & operating system

Windows 10

Affected Cores

No response

Environment information

No response

Relevant log output

[ERROR] [Thumbnail]: Download "H:\Emulators\RetroArch\thumbnails\Sony - PlayStation 2\Named_Snaps\Gran Turismo 4 (USA).png" failed: Internal error.

[bslenul]

Haven't tested myself but apparently it also breaks achievements and overlays: https://www.reddit.com/r/RetroArch/comments/1h9za1w/the_onscreen_controls_on_ios_have_been/ 😓

edit: Confirmed, tested with a N64 game, took ~20sec for achievements and overlay to show up...:

https://github.com/user-attachments/assets/e600cf4e-f152-432d-a75e-a66218cab1f4

[SyddG]

+1 on this. Currently experiencing it. Not able to access http://thumbnails.libretro.com.

[LeiFangBaby]

I have the same issue, and since it cannot access the site, manually downloading thumbnails/boxarts/titles doesn't work either, despite showing progress in chunks of 33% then 66% for each and taking a lot of time per-game.

I resorted to opening desktop UI with F5 and manually dragging the files for each, but it's extremely cumbersome and takes too long, as it doesn't support .jpg for boxarts and .png ones are only in those huge libretro console packs.

[warmenhoven]

Since all these things are tasks and there's only one thread handling tasks, any blocking networking operation will stall the entire thread. The on demand thumbnail downloader in particular is egregious about this, queuing multiple http requests. This blocks all networking, overlays, loading and saving states, etc.

I have some ideas for how to fix it, and I've started experimenting with a couple. Adding another thread is a potential option but without knowledge of what type of request it is, multiple blocking networking operations can still clog all the threads. Knowing what kind of task it is, would be a breaking api change to libretro, so not really feasible.

[warmenhoven]

Incidentally this exact problem happens on iOS all the time, when it's in airplane mode. It's something I've had in the back of my mind for a long time.

[LibretroAdmin]

This is indeed a known issue and definitely one of the big things that should be addressed in task_queue.c

[GlazedBelmont]

Getting a similar issue but I don't have playlists setup, could be cheevos related (as in, the same root cause also affects cheevos) but I can't reliably reproduce it. 1.20.0 - Android 13 - Device is a Retroid Pocket 5 though part of me wonders why I've only observed this issue on this device.

[warmenhoven]

The net_http refactor fixed this on threaded platforms by moving dns lookups to their own thread and caching responses (including failures).