WIP: Metal backend

[SamoZ256]

This PR implements a Metal backend for macOS. It should bring better performance compared to Vulkan, as well as fix some issues that wouldn't be possible with Vulkan due to MoltenVK limitations. I haven't tested many games, but those that I tried worked pretty well.

TODO:

• [x] sRGB
• [x] fix low resolution
• [x] proper present
• [x] fix black textures in sm3dw
• [x] buffer copies in a vertex shader
• [x] pipeline cache with binary archives
• [x] geometry shaders
• [x] GamePad screen in a separate window
• [x] fix texture readback
• [x] output shader
• [x] ImGui
• [x] Intel Macs support
• [x] proper surface copy
• [x] extend the shader hash
• [x] triangle fans
• [x] 1D shadow sampler emulation
• [x] instancing with mesh shaders
• [x] occlusion queries
• [ ] accurate occlusion queries
• [x] async shader compilation
• [ ] load pipeline binaries at startup
• [ ] async pipeline compilation
• [ ] fix CI

Screenshots:

New Super Mario Bros U

Captain Toad Treasure Tracker

Super Mario 3D World

Mario Kart 8

The Legend of Zelda: Breath of the Wild

Super Mario Maker

[Exzap]

Very impressive work so far! I did not expect anyone to take on adding a whole new renderer. From past experience when working on the OpenGL and Vulkan renderer I learned that the hardest problems come up at the end in regards to performance optimizations, weird API limitations which require workarounds or specifically for Metal: lack of geometry shaders. If the Metal backend gets past those problems into a fully functional state that is on par with the Vulkan renderer then we can talk about getting this merged and discuss any long term support plans. A proper full code review would also be necessary of course.

Looking forward to seeing this progress!

[Spritzerland]

this is some good work, thank you for it Samo

[SamoZ256]

Metal 3 based? Metal 2 don't have the requirements for emulating geometry shaders.

The concern would be that adding a third renderer would eventually lead to code rot, with fixes hitting the 2 windows/linux compatible api's and not the macos one.

I don't think I use any Metal 3 functionality. Regarding geometry shaders, they could be emulated with compute shaders, though I would much rather use mesh shaders and give up on Metal 2 compatibility.

[deReeperJosh]

I have had some issues testing with the Metal backend when launching into Lego Dimensions - the first time I launch the game with Vulkan it progresses fine (albeit with half the screen cut off), but then when I close and try to launch with the metal backend, it crashes and hangs halfway through the first menu/cutscene screens (as well as seemingly duplicating the menus to infinity in the top left of the screens). Then when I close and re-open and switch back to the Vulkan backend, it crashes/hangs at a similar stage during the first menu/cutscenes. Clearing shader caches and re-launching with Vulkan works sporadically, but most of the time I have to completely remove the Cemu folder in Library/Application Support in order to be able to launch with Vulkan again and not crash/hang. The Metal backend seems to always crash, no matter whether shaders are cleared or Library/Application Support/Cemu is deleted. Here are some videos of what I mean:

first launch with vulkan

then launch with metal

launch again with vulkan

Would love to help debug this as I normally do my dev work on Mac, and it would be excellent if I could test out my PR (https://github.com/cemu-project/Cemu/pull/1371) rebased on top of this branch, rather than needing to do my testing on my average Windows laptop

Edit: After enabling metal logging, I got a lot of "not implemented", and "Screenshots are not yet supported on Metal" so I guess I will need to stick with windows for the interim

[SamoZ256]

I have had some issues testing with the Metal backend when launching into Lego Dimensions - the first time I launch the game with Vulkan it progresses fine (albeit with half the screen cut off), but then when I close and try to launch with the metal backend, it crashes and hangs halfway through the first menu/cutscene screens (as well as seemingly duplicating the menus to infinity in the top left of the screens). Then when I close and re-open and switch back to the Vulkan backend, it crashes/hangs at a similar stage during the first menu/cutscenes. Clearing shader caches and re-launching with Vulkan works sporadically, but most of the time I have to completely remove the Cemu folder in Library/Application Support in order to be able to launch with Vulkan again and not crash/hang. The Metal backend seems to always crash, no matter whether shaders are cleared or Library/Application Support/Cemu is deleted. Here are some videos of what I mean:

first launch with vulkan

then launch with metal

launch again with vulkan

Would love to help debug this as I normally do my dev work on Mac, and it would be excellent if I could test out my PR (#1371) rebased on top of this branch, rather than needing to do my testing on my average Windows laptop

Edit: After enabling metal logging, I got a lot of "not implemented", and "Screenshots are not yet supported on Metal" so I guess I will need to stick with windows for the interim

Thanks for testing! The bug with duplicated frame happened on Minecraft for Wii U some time ago, but I think it got fixed. As for the Vulkan issue, other people have reported as well that Vulkan doesn't behave properly on the metal branch. I will have to investigate that. Do you possibly have a log for the Vulkan crash?

[deReeperJosh]

Thanks for testing! The bug with duplicated frame happened on Minecraft for Wii U some time ago, but I think it got fixed. As for the Vulkan issue, other people have reported as well that Vulkan doesn't behave properly on the metal branch. I will have to investigate that. Do you possibly have a log for the Vulkan crash?

I have a log but would it be more helpful if I enabled some more logging settings/run on the debug build?

metallog.txt vulkanlog.txt

[SamoZ256]

Thanks for testing! The bug with duplicated frame happened on Minecraft for Wii U some time ago, but I think it got fixed. As for the Vulkan issue, other people have reported as well that Vulkan doesn't behave properly on the metal branch. I will have to investigate that. Do you possibly have a log for the Vulkan crash?

I have a log but would it be more helpful if I enabled some more logging settings/run on the debug build?

metallog.txt vulkanlog.txt

Thanks for the logs. Debug build would probably give more info, but I see that the Metal log is full of screenshot request messages, so I am going to focus on that.

[deReeperJosh]

Built again with the debug version - the duplicated frame issue has gone away so must have not had some of your latest commits in, I am still finding the game hangs around the same place on Metal, even with the screenshot implementation, log attached: log.txt

[indianajson]

First of all, awesome work on this. We definitely needed a Metal implementation.

Seems the crash I mentioned earlier must have been an issue with my device, as it is now running the Metal engine smoothly. Awesome work!

[SamoZ256]

First of all, awesome work on this. We definitely needed a Metal implementation.

Seems the crash I mentioned earlier must have been an issue with my device, as it is now running the Metal engine smoothly. Awesome work!

Thanks! And exactly what game was crashing?

[SamoZ256]

Built again with the debug version - the duplicated frame issue has gone away so must have not had some of your latest commits in, I am still finding the game hangs around the same place on Metal, even with the screenshot implementation, log attached:

log.txt

Alright, good to hear that. I have no idea what could be causing the game to freeze though.

[indianajson]

Thanks! And exactly what game was crashing?

Paper Mario Color Splash. When I first launched the game using Metal FPS was 1-2 then when I got past the start screen it hard crashed (macOS froze), but after the restart I tried it again with Vulkan and it was fine so I decided to risk it and suddenly Metal was working. Still not sure if it was my device or Cemu.

[PotionBelt]

Amazing work on this project! I'm on an M1 MacBook Pro and gave your Metal backend a go with Wind Waker HD.

In general, things are much snappier than Vulkan with no graphical weirdness I can see, though the areas with many particles e.g. Dragonroost Island are still very choppy / low framerate (fluctuates between 8-15fps on my system). Other places with the same issue include the purple sparkles on the title screen and bomb explosion particles, which also tank the framerate.

Is there a fix for this in the pipeline? Would be happy to keep testing.

[SamoZ256]

Amazing work on this project! I'm on an M1 MacBook Pro and gave your Metal backend a go with Wind Waker HD.

In general, things are much snappier than Vulkan with no graphical weirdness I can see, though the areas with many particles e.g. Dragonroost Island are still very choppy / low framerate (fluctuates between 8-15fps on my system). Other places with the same issue include the purple sparkles on the title screen and bomb explosion particles, which also tank the framerate.

Is there a fix for this in the pipeline? Would be happy to keep testing.

Thank you for your feedback!

I have heard about the frame drop in Wind Waker HD. I don’t know if there is much I can do about it, but if you would make a frame capture (guide: https://github.com/SamoZ256/cemu-frame-capture), I could take a look at it and see if I could make a hack specifically for this game to improve it.

Also, is the frame rate better than Vulkan in the areas with many particles?

[PotionBelt]

Amazing work on this project! I'm on an M1 MacBook Pro and gave your Metal backend a go with Wind Waker HD. In general, things are much snappier than Vulkan with no graphical weirdness I can see, though the areas with many particles e.g. Dragonroost Island are still very choppy / low framerate (fluctuates between 8-15fps on my system). Other places with the same issue include the purple sparkles on the title screen and bomb explosion particles, which also tank the framerate. Is there a fix for this in the pipeline? Would be happy to keep testing.

Thank you for your feedback!

I have heard about the frame drop in Wind Waker HD. I don’t know if there is much I can do about it, but if you would make a frame capture (guide: https://github.com/SamoZ256/cemu-frame-capture), I could take a look at it and see if I could make a hack specifically for this game to improve it.

Also, is the frame rate better than Vulkan in the areas with many particles?

Any workaround would be amazing! Have Discord DM'd you the frame captures.

The frame rate is perhaps very slightly better in the particle-heavy areas on Dragonroost Island. With Vulkan that area is more like 8-13fps.

[darkuser89]

log.txt

thanks for the hard work. Unfortunately, Mario Kart doesn't start the second time

[SamoZ256]

log.txt

thanks for the hard work. Unfortunately, Mario Kart doesn't start the second time

Hi! This could probably (temporarily) be solved by clearing the shader cache. Also, make sure you are on the latest testing release.

[SamoZ256]

For anyone wanting to report game status/issues, please use this guide: https://github.com/SamoZ256/cemu-metal-game-compatibility.

[Exzap]

FYI conflicts are because of #1443 being merged

[Bentheminernz]

Is this Metal backend planning on fixing glitched text in some software?

[SamoZ256]

Is this Metal backend planning on fixing glitched text in some software?

I don't think that's a problem with the backend (though I am not sure)

[expensivecake290]

hi is work still being done to this for Mac ?

[SamoZ256]

hi is work still being done to this for Mac ?

I haven't done much work on it lately, but yes

[yannboyer7862]

Tank you for your great work ! Writing a whole new renderer for CEMU isn't an easy task. I genuinely hope you'll be able to finish it and that it gets merged, because MoltenVK is only a band-aid, not a permanent solution, especially since it doesn't geometry shaders and other important features.

I also saw that now CEMU has a new AArch64(ARM) Recompiler(someone wrote it with xbyak_aarch64), so combining this + Metal backend could make CEMU running natively on Apple Silicon Macs without Rosetta 2 and MoltenVK.

As a M1 Mac owner this would be amazing !

Take your time !

[SamoZ256]

Tank you for your great work ! Writing a whole new renderer for CEMU isn't an easy task.

I genuinely hope you'll be able to finish it and that it gets merged, because MoltenVK is only a band-aid, not a permanent solution, especially since it doesn't geometry shaders and other important features.

I also saw that now CEMU has a new AArch64(ARM) Recompiler(someone wrote it with xbyak_aarch64), so combining this + Metal backend could make CEMU running natively on Apple Silicon Macs without Rosetta 2 and MoltenVK.

As a M1 Mac owner this would be amazing !

Take your time !

Thank you for your comment, I appreciate it. As for AArch64, someone made a build that combines the arm recompiler with the Metal backend, and the results are amazing! I believe you could find a link in the Cemu discord

[Red6785]

I've been using the metal backend fork on my MacBook and I just wanted to say that the work that's been done is incredible.. I got better performance at lower power with better graphics. Thank you so much. Take as much time as you need.

[hauntek]

@Exzap Too strong, macOS 15.5 Sequia Metal API The Legend of Zelda: Breath of the Wild unlocks FPS++, can reach 50-60FPS, I can't imagine how strong the native Apple chip version will be! Compared to Vulkan, which can only FPS25-30, Metal API can achieve 50-60FPS without any issues with the graphics, and the display is very good!! I am looking forward to merging the main version!