Input lag of around 100-150ms (might be related to AppleTV, not Moonlight....)

[anuraagvaranasi]

Describe the bug There's about 100-150ms extra input lag when running Moonlight on AppleTV.

The way I'm testing is a bit "dodgy" but seems to be a good enough proxy; I'm running moonlight on many clients and pressing a key button on my PC keyboard. I'm then timing the difference in the input sound between my host computer and client.

Note: TV is on game mode.

My findings are

• AppleTV ethernet: 200+ms (usually 230-260ms)
• lgOS ethernet: around 100ms
• macbook wifi connected to TV HDMI: around 100ms
• macbook wifi on screen: unable to record (so quick I can barely hear a delay)
• iPad wifi on screen: unable to record (so quick I can barely hear a delay)

Tetsting makes sense IMO; it's about 80ms extra input lag from TV, rest of latency is unable to be determined due to network. AppleTV latency is the only anomaly.

Sadly it might be an AppleTV specific issue though...I did the same test with the SteamLink app and found similar latency with Moonlight (i.e. 200+ms)

Steps to reproduce N/A always on appleTV

Affected games List the games you've tried that exhibit the issue. To see if the issue is game-specific, try streaming Steam Big Picture with Moonlight and see if the issue persists there.

Other Moonlight clients

• Does the issue occur when using Moonlight on PC or Android? No

Moonlight settings (please complete the following information)

• Have any settings been adjusted from defaults? No
• Does the problem still occur after reverting settings back to default? Yes

Device details (please complete the following information)

• iOS/tvOS version: AppleTV with tvOS 17.5.1

Server PC details (please complete the following information)

• OS: [e.g. Windows 10 1809] Windows 11 22621.2361
• GeForce Experience version: [e.g. 3.16.0.140] 3.28.0.417
• Nvidia GPU driver: [e.g. 417.35] 556.12
• Antivirus and firewall software: [e.g. Windows Defender and Windows Firewall] Windows defender + windows firewall

[rgov]

Possibly related: There is some discussion about latency in #535, which hasn't been touched since 2022. @felipejfc measured 25-50ms of additional latency when streaming to an Apple TV 4K versus other devices.

[1Humza]

Same issue here. Statistics overlay doesn't catch the added latency either

[1Humza]

Maybe it's related to input. The latency disappears with hardwired peripherals connected to host PC while viewing stream on ATV. With Bluetooth peripherals I can feel the added delay. 🤔

[moi952]

Maybe it's related to input. The latency disappears with hardwired peripherals connected to host PC while viewing stream on ATV. With Bluetooth peripherals I can feel the added delay. 🤔

Yes it is unfortunately linked to the input, on apple tv the input is sent to each image instead of being sent directly, so you directly take 16ms of delay in addition. Ideally, you should use the Apple TV with a controller connected to the PC, or via a virtualHere (USB over IP)

[rgov]

Maybe so, but the input latency doesn't apply in my case. I was using a Bluetooth Xbox controller paired to the PC, with only the video and audio streaming to the Apple TV. I'm not especially sensitive to latency but the video lag made it impossible to aim in a FPS.

[liamdebeasi]

I'm running into a similar latency issue. In particular, I'm seeing ~100-150ms of latency when streaming frames back to the Apple TV client when using Apple TV 4K. Both my server and client are hardwired on the same network.

My (admittedly limited) understanding of the problem is this has to do with the usage of AVSampleBufferDisplayLayer as it has a display queue under the hood that we can't control. It looks like using VideoToolbox gives us more timing control which is presumably part of why https://github.com/moonlight-stream/moonlight-ios/pull/535 was opened.

Is there anything we can do to get https://github.com/moonlight-stream/moonlight-ios/pull/535 over the finish line? I'd be more than happy to help test and submit feedback.

[pierreski]

I'm also available to help with testing #535 to close out this issue as well! Just got an Apple TV 4K, and the amount of lag I'm experiencing in Moonlight is very noticeable.

• I experience no perceptible lag when using an Nvidia Shield, but the lag on Apple TV 4K is noticeable
• I've tried using both an Xbox one controller connected to both the Apple TV 4K and the host computer - noticeable lag in both instances
• My Apple TV 4K is connected via ethernet, so bandwidth / network latency is not the problem

[callephi]

Do we think this is something ultimately fixable through Moonlight? In my experience, there is a heavy piece tied into the Bluetooth module used in the Apple TV (4K 3rd gen 128GB in my case), as moving closer to the unit provided smoother input but alas still had very noticable latency, despite the claimed ~6ms through statistics in Moonlight.

[liamdebeasi]

It sounds like there are 2 issues:

1. Input delay with a bluetooth controller
2. Apple TV rendering delay when receiving stream data from sunshine

It might be worth having separate discussion threads for these as the fixes for both will likely be different. I can't speak to the bluetooth issue as I am not experiencing that, but it sounds like https://github.com/moonlight-stream/moonlight-ios/pull/535 could help reduce the rendering latency.

[rgov]

To quantify, one could set up a test like this:

1. Write a program that randomly cycles the screen color at 60 fps (a simple HTML page with some JavaScript is easy enough).
2. Attach computer directly to TV via HDMI, using screen mirroring.
3. Record computer screen and TV in slow motion. The iPhone does up to 240 fps.
4. Count number of frames between computer screen change and TV screen change. The TV refresh delay is n/240 seconds, basically our lower bound for delivering an image to the screen.
5. Attach Apple TV, a Mac, and Sunshine PC to same Ethernet network.
6. Stream from Mac to TV using AirPlay Mirroring and repeat timing experiment. Subtract TV refresh delay. This quantifies the network + video codec delay using a first-party implementation.
7. Stream from PC to TV using Moonlight and repeat timing experiment. Subtract AirPlay delay and TV refresh delay. This is any delay introduced by Moonlight / Apple's media streaming stack for third parties.
8. Optionally, repeat with a device like Nvidia Shield.

For Bluetooth delay it requires some coding. Maybe there's an easier way.

1. Set up a microcontroller like an Arduino to emulate a Bluetooth keyboard (or any HID device class). It should should wait for a signal over serial and then send a HID event.
2. Write a program on the PC that triggers the microcontroller and listens for the input event. Record timestamps of both events.
3. Run the experiment with the microcontroller connected via Bluetooth to the PC. The time delta is the lower bound for processing a button press on a Bluetooth controller.
4. Run the experiment again with the microcontroller connected via Bluetooth to the Apple TV. The time delta now includes delays introduced by the Apple TV Bluetooth stack, Moonlight's event processing, the network delay of relaying the event, and Sunshine's event processing.
5. Optionally, run the experiment again with Nvidia Shield to approximate the delay introduced by the Apple TV + Moonlight.

[liamdebeasi]

I ended up switching to using a Raspberry Pi 5 instead of an Apple TV for Moonlight. I'm able to stream at 1440p @ 60fps with very little latency. The gaming experience there has been really smooth, though it does require more setup than the Apple TV app does. I haven't been able to get it to work reliably with 4k @ 60fps, but I sit far enough from the TV that I can't really notice the difference between 4k and 1440p 😄

[PhilSpieltLOL]

I've been experiencing very similar major latency issues with the AppleTV Rendering Pipeline and the Input delay of Bluetooth controllers due to how tvOS handles Bluetooth devices. It seems like it's still an ongoing issue. I also set up a Raspberry Pi with VirtualHere which improved the latency a little bit, however there still is noticeable unfixable major latency just from the Rendering Pipeline of the AppleTV. I ended up buying a FireTV Cube which I will be using for 4k Streaming at 60fps for now, there now is almost no noticeable latency even with a purely WiFi setup. The current setup is a bit annoying though because the FireTV Cube is worse for everything else except Streaming via Moonlight since the OS isn't nearly as good as tvOS overall and it has noticeable color banding issues. That means that I now have to switch between those 2 devices because the AppleTV is a great device except for when it comes to Gaming.

I don't know if it's an Apple issue, a fixable Moonlight App issue or both, but it would be great if the issue could get fixed somehow.

I would be happy to help with anything I can or to test any possible fixes.