moonlight-qt
moonlight-qt copied to clipboard
moonlight-stream
Enable AV1 hardware decoding on M3 Apple devices
Describe the bug Hardware AV1 decoding does not work on M3 Apple devices and defaults back to software decoding if set to automatic
Steps to reproduce
- Install Moonlight 5.0.1 on any Apple M3 device
- Set up a sunshine host that can hardware encode AV1
- In settings, set force hardware decoding, choose AV1 as the video codec
Screenshots
Moonlight settings (please complete the following information)
- Set force hardware decoding
- Choose AV1 as the video codec
Client PC details (please complete the following information)
- OS: macOS Sonoma 14.1.1
- Moonlight Version: 5.0.1
- GPU: Apple M3 Pro
Server PC details (please complete the following information)
- OS: Windows 11
- GPU: RTX 4090
- Sunshine Version: 0.21.0
Additional context Apple added AV1 hardware decode support to M3 devices: https://www.apple.com/newsroom/2023/10/apple-unveils-m3-m3-pro-and-m3-max-the-most-advanced-chips-for-a-personal-computer/
We use FFmpeg for video decoding, and it doesn't yet support AV1 decoding using the macOS VideoToolbox API yet.
Once it does, we can pull in the new version and it will just work without any work from us.
Thanks, just found the ticket (10642) where FFmpeg is tracking this. I’ll keep an eye on the progress there and comment here again when they’ve added that feature to a release
We use FFmpeg for video decoding, and it doesn't yet support AV1 decoding using the macOS VideoToolbox API yet.
Once it does, we can pull in the new version and it will just work without any work from us.
apparently this branch implement AV1 for m3 macs :
https://github.com/jeeb/ffmpeg/commit/2a179d586f807e552e86967d07573f8ea44bacfd
WIP avcodec/videotoolbox: add AV1 hardware acceleration
could we build moonlight with this ffmepg patch ? git/cmake/submodules shenanigans 🤔
Getting 5-6ms at 150mbps HEVC 10 Main HDR with hw decoding forced Mac M3 Max 10+4 CPU, 30C GPU
isn't it kinda slow ?
@Foul-Tarnished We build our own FFmpeg, so we can include custom patches if we want. However, I never saw confirmation anywhere that the changes mentioned in that branch actually worked. I just saw that one report of a crash and no users reporting working AV1 decoding.
Currently working on sending a patch to ffmpeg. Got this to work. https://trac.ffmpeg.org/ticket/10642#comment:10
♥️ Let me know if you need testing on real mac m3 hw, once you have a build
Le dim. 5 mai 2024, 18:08, Ruslan Chernenko @.***> a écrit :
Currently working on sending a patch to ffmpeg. Got this to work. https://trac.ffmpeg.org/ticket/10642#comment:10
— Reply to this email directly, view it on GitHub https://github.com/moonlight-stream/moonlight-qt/issues/1125#issuecomment-2094862974, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZVYVBRNSPNIZXD7B52PNHTZAZKOLAVCNFSM6AAAAAA74WIJ42VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJUHA3DEOJXGQ . You are receiving this because you were mentioned.Message ID: @.***>
♥️
Let me know if you need testing on real mac m3 hw, once you have a build
Le dim. 5 mai 2024, 18:08, Ruslan Chernenko @.***> a
écrit :
Currently working on sending a patch to ffmpeg. Got this to work.
https://trac.ffmpeg.org/ticket/10642#comment:10
—
Reply to this email directly, view it on GitHub
https://github.com/moonlight-stream/moonlight-qt/issues/1125#issuecomment-2094862974,
or unsubscribe
.
You are receiving this because you were mentioned.Message ID:
@.***>
Yeah, okay. Yesterday I pulled the code for the moonlight and going to build it against my patch to see if it works 'as-is'. Sending the patch to ffmpeg gonna take some time I guess, from review up to an official release(or a commit into a master). So I guess I would just build a version for a test usage. Will link a branch if any change to a code will be added.
Got this to work in moonlight, however needed to do a little hack on ffmpeg's side. Need to take into account a few things in terms of copying sequence header obu into the config format of VT. But yeah, it works 'as-is', no changes needed to the moonlight code. One thing that I noticed is that I don't see any noticeable difference between hw accelerated vs sw accelerate using libadv1d. The decoding time per frame is basically +- the same, the energy consumption is the same as well. Will make a few more tests for this.
updated: ok no, idk but I suppose my power draw metrics are wrong; that being said when using hw acceleration the cpu usage of moonlight is significantly less, 20-25%(hwaccel) vs 115-125%(sw libdav1d) however its worth noting that the decoding time shows the opposite relation I am getting 4-5ms per frame(1920x1080x60) on sw libadv1d vs 6-7ms per frame on hwacceled Have no idea why is that at this moment, going to dig a little bit into this.
Here is a link to a branch of mine:
https://github.com/nenkoru/FFmpeg/tree/videotoolbox_av1_nenkoru
You could build it yourself and just copy the libavcodec.61.dylib in place of the one in ./libs/mac/lib after pulling. After that just build the code and it would work.
I could also attach a libavcodec.61.dylib itself or even a built application itself, but I need a clear vote for that.
Hey, has this been merged? Just the other day I was trying this out and getting the message saying that the software decoder would be used instead; Today, however, I got a different message, saying that my GPU did not support AV1 Main10 HDR streaming. The stream then started with AV1, but without HDR on the client (image was blown out due to HDR/SDR mismatch).
Is that really the case? Will HDR over AV1 not be supported on M3 devices?
I'm using Moonlight 6.0.0 on a Macbook Pro (M3 Pro).
I read that AV1 decoding (with HDR support) on Moonlight is supported on iPhone Pro 15 as well as the new M4 iPad Pro. Will it also be released for the M3 Pro macOS version?
Can anyone check and confirm if AV1 with HDR works on the M3/M3 Pro models?
Can anyone check and confirm if AV1 with HDR works on the M3/M3 Pro models?
See my last comment above