(WiiU) Update WUT and add RetroArch target for WUHB files (Aroma enviroment)

[ShadowOne333]

Description

A few weeks ago, the WiiU received a new form of CFW called "Aroma", which runs apps under the WUHB extension. What this is effectively creating a coldboot option for the system, and also adds a bunch of features and other things as plugins directly on boot.

One of said options is the ability to be able to run Homebrew apps straight from the WiiU's home menu, as if it they were official apps on the system. This way, each homebrew app can be threated as their own app within the system.

In order to make RetroArch compatible with this setup, an update of WUT is required, as well as having a TV logo image (tv_splash.png, 1280x720), the gamepad image (drc_splash.png, 854x480) and the home menu image (icon.png, 132x179). With that, creating the WUHB file out of the rpx should be easy enough by following these recommended instructions after updating both wut and wut-tools: https://gbatemp.net/threads/release-aroma-environment-for-wii-u.618474/page-17#post-9940681

Expected behavior

Updating to the Aroma-friendly WUHB format would allow for RetroArch to be recognized and properly loaded on the system, alongside their proper boot logo, icon, and message.

Actual behavior

RetroArch currently cannot run under this environment since; while Aroma does have support for RPX apps, upon loading it simply stays on a black screen.

[vgmoose]

Maschell added the following info in the Aroma discord about rpx-loading:

Retroarch currently loads a core by loading into a special area in the memory and then using some old HBL magic to launch it. With aroma we can't use this region, because it's reserved for aroma and it's plugins. You can load a .rpx via Aroma though, Mocha_LaunchRPX is too "low" level, but librxpload (https://github.com/wiiu-env/librpxloader) could be extended to load a different .rpx which is inside the .wuhb.

[grockk]

I wonder if the vWii retroarch could be updated to support the sign_c2w_patcher in a forwarder or inject. We'd be limited to 480p but for retro games on Wii U that seems like it'd work pretty good enough. And then it would be compatible with Aroma.

[ashquarky]

I think I'm going to have a go at a wut port, at least depending on my availability over the holidays... I wrote up a bit of a game plan that might work but there are a few key questions:

• Merging changes to all the cores and the frontend and the docker image at the same time is going to be a pain. Do we try and make a new platform= (wiiu-wut alongside wiiu?) to ease that or do we just wait for a release and then merge immediately after? I would prefer the latter but I don't know how RetroArch's release cycle works
• Should wut be added in-tree? Since all the builds run on Docker now I don't see a reason to (especially considering that wut versions are somewhat tied to devkitPPC versions) but since libogc is in-tree it's worth asking

cc @LibretroAdmin @gblues

[LibretroAdmin]

wiiu-wut - as in a new platform/target - seems like a good idea. Overall I'm fine with whatever @gblues thinks we should do.

Should wut be added in-tree? Since all the builds run on Docker now I don't see a reason to (especially considering that wut versions are somewhat tied to devkitPPC versions) but since libogc is in-tree it's worth asking

How big is it ?

[nstrong-scw]

I would prefer that we avoid adding WUT in-tree. We should consume WUT artifacts, and we can control the version we get without resorting to completely duplicating the source tree. Adding it in-tree creates a bunch of toil. If we needed to add functionality to WUT, it should be contributed upstream. Which yes, may take a little longer, but it means that more projects than just RetroArch benefit.

I am in favor of having a separate build target. I propose that we just call it wut (which already implies wiiu). I also propose that once all cores have successful wut builds that the wiiu target builds be left deprecated. This is because the whole point of WUT is to provide a stable dev toolkit and I have no interest in maintaining 2 builds indefinitely.

[gblues]

whoops, I was signed into my work github account. ^ that's me. sorry!

[ashquarky]

I will mention that my current draft of RetroArch (frontend) does not keep both codepaths - it assumes wut.

Also, the differences in the cores might not be that bad - snes9x at least was just some (backwards-compatible) compiler flag changes; thus for that core a new target isn't needed. I assume other cores will be more complicated though.

(specifically -mwup needs to be replaced - I swear I've PRed this to some cores before but I can't find it now..)

[ashquarky]

For those following along:

• RetroArch tree: https://github.com/ashquarky/RetroArch/tree/wiiu-wut
• toolchain (docker): https://git.libretro.com/quarky/libretro-build-wiiu/-/tree/wut

Right now it mostly works under Tiramisu, aside from some long load times. Toolchain container is not well-tested, and cores still need to be updated.

[ashquarky]

Just submitted the makefile changes for snes9x - with luck, all the cores will be like this. The core now builds fine on both old and new devkitPPC (with and without wut) at the cost of some extra meaningless warnings on the old version.

[ShadowOne333]

I didn't see the last messages, this all sounds fantastic. Did the recent commit that got merged for the makefile changes add .wuhb cores for RetroArch on WiiU? Or is that still WIP?

[ashquarky]

As an update for everyone:

• Basically everything is working except for Salamander, which bootloops for some reason
• I've had to disable libfat for the SD card since many Aroma modules mount the SD using the Cafe driver, and it's unsafe to have two drivers mounting the same filesystem at once. This makes startup really slow so I may re-enable it on Tira only. Playlists might be broken due to sd: becoming fs:/vol/external01.

I didn't see the last messages, this all sounds fantastic.

Did the recent commit that got merged for the makefile changes add .wuhb cores for RetroArch on WiiU?

Or is that still WIP?

wuhbs aren't in for this initial porting stage, though it may end up being a solution to the loading time issue.. we'll see! I'll probably try and land initial support first.

[ShadowOne333]

As an update for everyone:

* Basically everything is working except for Salamander, which bootloops for some reason

* I've had to disable libfat for the SD card since many Aroma modules mount the SD using the Cafe driver, and it's unsafe to have two drivers mounting the same filesystem at once. This makes startup really slow so I may re-enable it on Tira only. Playlists might be broken due to sd: becoming fs:/vol/external01.

I didn't see the last messages, this all sounds fantastic. Did the recent commit that got merged for the makefile changes add .wuhb cores for RetroArch on WiiU? Or is that still WIP?

wuhbs aren't in for this initial porting stage, though it may end up being a solution to the loading time issue.. we'll see! I'll probably try and land initial support first.

That's awesome! Really looking forward to what you come up with for the RA rework for WiiU. Hope it all goes well with the last tidbits of nuances

[Clownacy]

• I've had to disable libfat for the SD card since many Aroma modules mount the SD using the Cafe driver, and it's unsafe to have two drivers mounting the same filesystem at once. This makes startup really slow so I may re-enable it on Tira only.

That sounds familiar: devkitPPC's newlib port had its disk IO buffering disabled for the longest time, and it has only been re-enabled in a recent commit. Without buffering, SD card accesses are extremely slow. No update to devkitPPC has been released yet with this fix, so in the meantime IO buffering has to be explicitly enabled with setvbuf(fp, NULL, _IOFBF, BUFSIZ);. I've done this before in a port of mine, and it does greatly improve the SD card access times (including making the homebrew start-up much faster).

I hope this helps.

[ashquarky]

It's a good shout - setvbuf does help a lot, and we're already doing it in the wiiu port. The issues are instead mostly from the hundreds of small files existing in the assets folder, as well as the directory listings RA does. (the dirent functions get slower if there are larger files in the folder? not more, but larger?)

...wait, why are we setting a 128k vbuf and then a 4k one straight after? that's not in a3be192, did we get screwed on a merge conflict?

[ashquarky]

Also hey, had no idea devkitPPC had enabled buffering upstream. Do we know what their buffer sizes are and what release rev that change is in? Very cool if they've fixed one of the biggest pain points for porting things (fgetc)

[Clownacy]

From what I can make of newlib's code, the default buffer size is 1024 bytes (BUFSIZ). There doesn't appear to be any release of devkitPPC that has this change yet: buffering was enabled in August of last year, but the most recent release of devkitPPC is from May.

[ashquarky]

Makes sense. Looks like we'll still have to setvbuf in RetroArch (64k and 128k are the ideal sizes for large files in Cafe) but it'll be a nice help for other projects.

[Ploggy]

Hey all. I've been chipping away at editing the Retroarch Cores Makefiles to add the necessary changes Quarky mentioned for Aroma support. I have changed all the Cores that are listed here: https://github.com/libretro/RetroArch/pull/14925

I manged to compile a handful of modified Cores and have tested them on WiiU to see how they perform. There are a few Cores that fail to compile with the new changes but I guess that's to be expected and will have to be fixed on a per-core basis? Good news is they do boot direct from the WiiU Menu and can load a rom :) Bad news is Core switching seems to be broken and loading a rom from usb will throw an error.

Another problem (as mentioned above) is the long core load/rom load times, it can take up to 2 1/2 mins from loading the Core from WiiU Menu to game boot. https://www.youtube.com/watch?v=jn8YIVMi4EA&feature=youtu.be

But even with current issues is great to see Retroarch Cores working in Aroma! the future is now! :D Thank You Quarky!!!

[Sowden]

Hey @Ploggy, is there anywhere we could download this experimental wuhb to try it out? 😁

[Ploggy]

https://www.mediafire.com/folder/bvrennhlwsw2s/Aroma+Cores It isn't in Wuhb format just single rpx files but sure :) I did create a Wuhb file for testing but Retroarch hasn't got the ability to load cores from within a packaged Whub.

EDIT: fixed link

[Sowden]

Thanks for the speedy response! Editing my first post cause I'm dumb, lol. I'll work with the Atari cores to see how well it will work, but it is nice to see it running in Aroma. Good job!

[Ploggy]

The Atari Cores should be fine, just be warned the Cores do take a while to boot currently and so does rom loading ;)

[ashquarky]

Cores do take a while to boot currently and so does rom loading ;)

I am pretty sure this is an Aroma-specific issue, by the way. I don't have a great fix for it right now though. If you're switching between Aroma and Tira, try deleting retroarch.cfg and see if that speeds it up - it'll remember that libfat was disabled (from Aroma) otherwise ;)

[Ploggy]

(Testing Amstrad CPC Core) On Tira CFW = Core swapping possible.. loading a Aroma Core in Retroarch takes 41 seconds Once that Core is loaded, loading a game takes a further 41 seconds

On Aroma CFW (with fresh Retroarch.cfg) = Core swapping not possible.. When you try to load another Core after 54 seconds it'll return back to the Retroarch UI but the last Core will still be loaded. loading Core from WiiU Menu takes 1 min 18 seconds. Once that Core is loaded, loading a game takes a further 55 seconds.

So its a problem for both CFW but on Aroma Core switching is broken as well.

[ashquarky]

Core switching was definitely working in my tests :( Guess I'll have to check again...

[Ploggy]

Closing content and browsing long romlists with thumbnails is alot slower too :( Its a similar situation we had in Retroarch a while back with slow io?

[ashquarky]

It's really odd that you're still seeing it in Tiramisu. You've tested Tira, with everything up to date off tiramisu.foryour.cafe, and a clean retroarch.cfg; and it's still much slower than the current nightlies?

[Ploggy]

I can recheck tira updates. gimme sec.

[Ploggy]

The Cores are relatively recent like a week old and the Retroarch I'm using is your wiiu-wut-merge.. Even with 100% up to date Tira and fresh Retroarch.cfg it still takes a while to boot core/rom?

[ashquarky]

Alright, thanks for checking. I'll have another look :/