libretro
[3DS] [Feature request] XMB Performance on 3DS
Description
On my New 3DS, when starting up RetroArch, opening a game, closing a game, and switching cores, it spends around 1 minute on a black screen when using XMB, whereas on RGUI it only takes a few seconds.
I assume that this is because of the horrible SD random access IO speeds, and there are 1421 files in the assets/xmb folder, despite all of them combined taking up only 4.1 MB. Could there be an option to package these assets as a .tar and during startup, temporarily load them into RAM?
Alternatively (or additionally), it would be great if it would just have to load this once, rather then reloading it when switching games/cores/back to the menu. Since it's statically linked, I'm not sure if this is an option however
Expected behavior
RetroArch loads quickly
Actual behavior
RetroArch takes a long time to do any of these tasks
Steps to reproduce the bug
Open RetroArch
Bisect Results
N/A
Version/Commit
N/A
Environment information
N/A
If anything, I believe using a .tar to compress all the assets will make loading slower, because it would have to decompress the assets.
XMB is rather graphic intense. You might want to try disabling the background shader and such.
.tar doesn't compress anything, and the problem is the amount of tiny files...
This issue is about loading performance on XMB. Tar won't help loading performance, since it would still load the same amount of data from the disc. XMB is pretty hefty, so I'm not surprised it's slow. 800 × 240 is also pretty tight resolution for it too.
Try a faster SD card? Disable all the shaders?
I'm not sure what the new title means, XMB is already supported so I'm not asking about adding it...
Try a faster SD card? Disable all the shaders?
I think the bottleneck is the actual console, because on a computer I can quickly copy these small files
Tar won't help loading performance, since it would still load the same amount of data from the disc.
As a comment in a similar issue put it:
In order to actually use a core, we need to read its corresponding info file (this tells us, among other things, the type of content it can load). These are teeny tiny files, and on every other platform it takes a few milliseconds to read them - but 3DS is again a special case. It has the most appallingly slow microsd random access/transfer speeds I have ever seen anywhere - it's barely functional. So something that takes no time at all on every other platform takes forever on 3DS.
The current method of loading individual info files is pretty lousy from a performance perspective (the config file format is hugely inefficient when it comes to parsing operations, and multiple file open/close operations are bad in general, even if they are fast on other platforms). A single file would improve this, but:
The 3DS can read a medium-sized 4 MB file relatively quickly, but when trying to read a lot of small files (hundreds in this case), because of the slow access speeds, and inefficiencies of the FAT32 filesystem, it takes a long time to my understanding.
There's an approach regarding loading a lot of graphics/sprites quickly on 3DS, used commonly by for example games made using LövePotion engine (which is Love2D game engine ported to 3DS), but probably many commercial games as well. You essentially make a single texture atlas (or spritesheet, which is an older term) file that includes all of the graphics and than you use an algorithm that extracts and loads graphics from this file. From what I understand similar approach is also used for many websites that want to minimize network load, and make websites load faster, but here it could potentially remove the performance hit of loading many small files from a FAT32 partition on the SD card. Is this possible to implement in RetroArch?
I'd imagine it would be simpler to just extract a .tar into RAM
I agree. The 3DS performs very poorly for large amounts of reads, regardless of the size of the file.
I have previously created a cache for about 60 info files that are read before core is executed in one file, and this improved the performance of the run from 29 seconds -> 12 seconds. (maybe it's back to loading in 1 minute now, unlike 3 years ago) https://github.com/libretro/RetroArch/pull/12350
The resource size of the xmb is small, but it is composed of multiple files, so I think changing it to a tar or texture atlas would be a really good performance improvement.
