Genesis-Plus-GX
Genesis-Plus-GX copied to clipboard
libretro
[Feature Request] Soft Patch roms (non-cd)
It would be great to be able to soft patch non-cd roms, or at least expose an option so you can soft patch at your own risk.
While such a thing may be possible for some cores, as I recall this core needs the actual location of the file regardless of what type of media is being loaded.
Well, in theory, since soft-patching is likely only usable for ROM files (not CD image files) and since loaded file name is apparently always available from libretro, it could be possible to modify the "load_archive" function in libretro.c to use the ROM pointer/length provided by libretro API and initialize the ROM buffer required by the emulator using libretro provided ROM pointer (instead of reading it again from file location), while still returning the ROM file extension and ROM size expected from this function. Note that this function is only called by the emulator (through load_rom core funtion) if the loaded file is a ROM file, not if it is detected as a CD image file.
As long as the loaded file path is available from libretro and the emulator can still open and read from the file to detect (then later emulate) CD image files, it does not matter to the emulator if the file was initially loaded (and patched) into RAM by the frontend.
From my understanding, the only problem is that libretro requires the core to set "need_fullpath" setting to FALSE to allow soft-patching and return a valid ROM data pointer (see original issue regarding soft-patching https://github.com/libretro/Genesis-Plus-GX/issues/54) but I don't know what will happen if this set to FALSE by default and the user tries to load a CD image file on platforms that don't have enough RAM to hold the entire file.
If the frontend behavior is to return a load error in this case then this means this will break CD support on any platform with limited RAM ressources and that we cannot have softpatching AND sega CD support on such platforms unless:
- a core option is added to selectively enable/disable "need_fullpath" option on the fly or
- frontend default behavior is modified to not return an error when"need_fullpath" is set to false and there is not enough RAM to hold the loaded content but instead behaves as if "need_fulllpath" was set to TRUE and inform the core (easily done by returning NULL into data pointer a). It would then be up to the core to decide if content can be correctly handled or if it should return an error to the frontend when retro_load_game is called. or
- frontend is modified to force loading of content into RAM if soft-patching is required by the user and still provide a valid *data pointer in this case that the core can use, even if "need_fullpath" is set to True
All 3 solutions would require (minimal) changes on Genesis Plus GX core side (limited to libretro.c).
Solution 1 is not very practical (requires user to change core setting when he wants to use soft-patching and when he wants to load CD games) and might not even be possible without reloading the core each time depending on when retro_get_system_info is called by the frontend.
Solution 2 looks fine but since it requires core to handle the cases where *data pointer is NULL, this might cause some unwanted crash when loading content that do not fit in RAM (while currently this would be handled by the frontend with a clean error exit) on some cores that would assume it is always valid because they have set need_fullpath = FALSE and therefore could possibly not check it is indeed different from NULL before using it.
Solution 3 seems the best to me as it does not require any API changes (only some precisions added in libretro.h API documentation) and likely remains compatible to existing core that use "need_fullpath=TRUE" since they were supposed to ignore *data pointer so far anyway. Those cores could still be modified (like GPGX) to check if *data pointer is not NULL so they know that they can use it instead if they want. If someone tries to soft-patch a CD image file, the result would either be a patching error if there is no enough RAM to hold the file (but the game could be loaded normally into the core if "need_fullpath" is set to true) or, if there is enough RAM, *data pointer would be set and it's up to the core to use it for the emulated CD or disregard it if is hard-coded to read from CD files.
It would be really nice to have. There are a lot of people who prefer soft patching to preserve the integrity of the original rom, and not having to fork rom sets
I think everybody understand why soft-patching is interesting and desired, it's not the issue here.
As explained above, the required change in genesis plus gx libretro portion to use patched ROM data instead of reading from ROM file are minimal and quite simple (I will try to push a commit in my repository to show the needed changes, even if it will have no effect without retroarch being fixed as well). The problem is that the frontend (rcetroarch) ONLY patches ROM data if the core asks him to ALWAYS load entire file content into RAM and it's not possible to tell the frontend to "do this only for ROMs, not CD image files" as the frontend has no way to distinguish between those, so whenever you will try to load a CD game, Retroarch will fail loading it unless you have enough RAM (Sega CD games can be up 600 MB).
I also tried solution 1 above (to make it user configurable) but it does not work as core settings are only available once a game is loaded.
So, unless I'm missing something (NB: this is a call to libretro devs to correct me if I'm wrong) the only solution is to also modify the frontend (retroarch) using either solution 2 or 3 above.
Yeah, I believe you're correct. #3 indeed seems most doable, just at a glance.
I have submitted a commit in my repository (https://github.com/ekeeke/Genesis-Plus-GX/commit/81becffb6c9d1ee9c754439eb4242633a5ae9076) with the modifications required to support ROM files already loaded in memory by Retroarch.
For the moment, since the core still sets "need_fullpath" to TRUE, Retroarch never attempts to load ROM file into memory and disable auto-patching so provided info->data is always NULL and the added code to support soft-patched ROMs will actually never be called.
Therefore, it still requires frontend modification (either solution 2 or 3 above) to be fully functional.
NB: I quickly tested it worked by compiling a build with "need_fullpath" set to False and verified the loaded file was indeed loaded into memory. As already explained, this solution is not viable as it requires up to 600 MB of RAM for Mega CD games, and thus would make the core unusable for Mega CD emulation on many platforms supported by Retroarch (and even on platform with enough RAM, forcing such high RAM use is not recommended practice unless explicitly enabled by the user).
Adding an option in the libretro core to preload game into memory would solve this problem by linking soft patching to that option. RAM usage shouldn't be a concern then for roms or CD's, as long as the option is disabled by default.
As explained above, adding a core option is not possible as, by design in retroarch, core settings are not accessible until game file has been loaded so it's already too late.
As explained in linked pull request, the solution requires adding an extension to libretro API to allow cores to enable soft-patching for some files (based on file extension) even when need_fullpath is set to TRUE by the core.
In the case of 64-bit builds, at least for Windows and Linux, it can just be loaded into memory regardless of extension since amount of memory isn't going to be a concern for those platforms since the average user there has at least 8GB of RAM, so regardless of whether it's a rom or CD, there should be no issues, need_fullpath can then just be set to false.
If these are still a concern for you, a temporary solution could be to provide a build with the above mentioned here or submitted to the Retroarch team as an experimental build until a proper agreed-upon solution can be found for all.
You are free to build your own core version with need_fullpath set to false if you cannot wait for a proper solution on retroarch side but I would definitively not release such build myself or force this for windows users assuming they have enough free RAM.
That's why I suggested a build posted here or released explicitly as an experimental build, not to replace the current master builds. It would also help to get additional feedback testing in case there are other unforeseen issues. I think most users would appreciate the thought given the lack of soft-patching support for this emulator, and given the fact that any necessary changes on the Retroarch side are not likely to happen any time soon.
Building with need_fullpath set to false does not load game, rom or CD, into memory, nor does soft-patching work with your recent changes to master. There's more going on here than a simple flag change during compilaton, whether that's more significant changes needed to your core, or Retroarch completely ignoring the need_fullpath flag state.
I managed to get the build working with need_fullpath set to 'false' and softpatching is now working.
Just a for your info, the reason it didn't work before was because the makefile needed to be cleaned first, despite being a clean install and fresh build environment.
For anyone that's interested, here is the build with softpatching functional, Windows 64-bit.
(note that all games, roms and cd images alike, will be loaded into memory, so make sure you have at least 4GB, more for multi-cd games loaded with a .m3u file)
Core based off of ekeeke's latest master here.