moonlight-qt
moonlight-qt copied to clipboard
moonlight-stream
stutters every so often on macOS
Describe the bug Stream is perfect but every so often, maybe around 5-10min, video/audio stutters for few seconds. The controls still work during this time, but stutters and skips until it's over. Then it's flawless streaming until the next stutter.
Moonlight settings (please complete the following information) I have tested with all variant of settings, changing Video bitrates from 20 ~ 150, v-sync, frame pacing, etc.. Occasional stutter still remains.
Client PC details (please complete the following information)
- OS: Mac OS X 10.14.2
- Moonlight Version: v0.8.0
Server PC details (please complete the following information)
- OS: Windows 10 1803
- GeForce Experience version: 3.16.0.140
- Nvidia GPU driver: 417.35
Moonlight Logs (please attach) Moonlight-1546797780.log
I've also seen this periodically on my MacBook, but it doesn't happen on either of my other two non-Mac computers streaming over the same WiFi network. Mine is the Early 2016 MacBook. What model is yours?
My guess is that it's something macOS is doing in the background that's eating up either CPU or network resources. Moonlight doesn't do any periodic work while streaming like that, so I can't imagine what it could be inside Moonlight. I'm going to try to catch the stutter in the Instruments profiler and see what's going on with the system at the time.
Thanks, I did some testing before your post. I had run the stream and concurrently ping against the host computer and surprisingly the ping did not loose packet at all during the stutter.
369 packets transmitted, 369 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.522/8.366/140.319/12.945 ms
This just reconfirms that it might be a macOS thing and not network inefficiency. Thanks!
I wouldn't totally trust the ping results. Ping is sending/receiving about 1 packet per second, while Moonlight (depending on how much is happening on screen) can be receiving 1,000 packets per second or more. If the network was only dropping a fraction of the total packets, the odds that one of the dropped packets would be your ping packet in all of that traffic is very low.
I captured a trace of it in Instruments System Profiler and found some interesting stuff:
Timestamp Narrative
00:05.286.573 Called "recvfrom(fd:45, buf:0x3ab2200, len:1.37 KiB, flags:0x0,...) = 0 Bytes" for 100.39 ms, end-of-file was encountered.
00:05.286.583 Blocked for 100.30 ms (99.9% of recvfrom's duration) starting at priority 47. Thread is waiting for event/lock with id 0xa33ba8a859006e9f. __recvfrom ← (6 other frames)
00:05.386.880 The thread was made runnable by a timer expiration (handled by CPU 0's timer interrupt handler). It waited for an available CPU for 66.54 µs (0.1% of recvfrom's duration) before running again at a priority of 47.
00:05.386.947 Ran for 51.42 µs on CPU 0 at priority 47
00:05.386.976 Called "recvfrom(fd:45, buf:0x3ab2200, len:1.37 KiB, flags:0x0,...) = 323 Bytes" for 52.55 ms
00:05.386.998 Blocked for 52.51 ms (99.9% of recvfrom's duration) starting at priority 47. Thread is waiting for event/lock with id 0xa33ba8a859006e9f.
So it really appears that recvfrom() is blocking for over 100 ms without receiving a packet. The Moonlight receive thread is not being preempted during that time or anything, since it's able to run and call recvfrom() again when the first call times out.
This confirms the issue is not our fault and lies with the network or OS network stack.
Wow interesting. I wonder if other real-time streaming platforms (PS4 remote play) would have the same problem or if there is a workaround.
Just want to report some of the things I've tried to resolve the issue that didn't work:
- reset NVRAM
- disable spotlight indexing (adding disk to privacy)
- disable most settings in 'energy saver'
- removing 'mediaanalysisd' daemon from startup source
This whole thread is very insightful. And I've had the same issue for what it's worth. But, it's also worth noting that I get this issue on iOS as well. Both on my 2016 iPad Pro and iPhone 6s Plus.
During the stutter, there is still interim packets of video/audio that was received. So it seems like recvfrom isn't entirely blocked..
If @euroclydon37 has problems on iOS too, does it mean the mac version and ios version utilize the same framework?
In the trace I recorded, I saw recvfrom() take over 150 ms to return a single packet at one point. This is consistent with the stutter observed, because we are getting a bunch of data dropped and the rest delayed significantly.
I'm highly suspicious of jumping to conclusions that iOS may be affected. Moonlight iOS and PC have totally different codebases and I haven't personally seen iOS affected myself.
We need to remember that 99.9% of the time these issues really are transient network problems that have nothing to do with Moonlight. It takes pretty extraordinary evidence to prove that it's not the network's fault when we can prove that packets are being delayed or dropped before reaching the client. I'm not even totally convinced this isn't what I'm seeing on my own network.
Very short network congestion conditions arise all the time, especially on WiFi with multiple clients and shared airspace, and real-time remote gameplay is about as intolerant of application to an imperfect network as you can find. In many ways, you are really seeing and hearing your network's performance. When packets are delayed, you see the video run in slow motion. When packets get dropped, you hear the audio crackle as it runs out of audio to play and see the video freeze until the congestion passes.
So it looks like the packet drop from the recvfrom blockage has something to do with the network layer for OS X, right? I was just wondering if OSX and iOS share the same network layer framework, thus creating similar issues. It's possible that the internal implementation is different though. Sorry, pure speculation on my part since I don't know moonlight codebase.🥉
However, I am going to test it out further on the same machine:
- PS4 remote play
- Windows 10 bootcamp
I'll report back the findings later this week.
Did some snooping around with packet capture on the server and client, found that during the hiccup, the packets were unable to reach port:
src dest
192.168.1.20 192.168.1.100 ICMP 70 Destination unreachable (Port unreachable)
And server tries to initiate a new TLS handshake:
src dest
192.168.1.100 192.168.1.20 TLSv1.2 916 Server Hello, Certificate, Certificate Request, Server Hello Done
192.168.1.20 192.168.1.100 TLSv1.2 1363 Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message
192.168.1.100 192.168.1.20 TLSv1.2 105 Change Cipher Spec, Encrypted Handshake Message
@kujohn do you have the full packet capture of those packets (including port numbers and the original packet that the ICMP message was referring to)?
There should be no HTTP/HTTPS activity from Moonlight after the stream is established until the session ends.
Is this what you're looking for?
Internet Control Message Protocol
Type: 3 (Destination unreachable)
Code: 3 (Port unreachable)
Checksum: 0x47fe [correct]
[Checksum Status: Good]
Unused: 00000000
Internet Protocol Version 4, Src: 192.168.1.100, Dst: 192.168.1.20
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0xe0 (DSCP: CS7, ECN: Not-ECT)
1110 00.. = Differentiated Services Codepoint: Class Selector 7 (56)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 1436
Identification: 0x3bb1 (15281)
Flags: 0x0000
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
...0 0000 0000 0000 = Fragment offset: 0
Time to live: 128
Protocol: UDP (17)
Header checksum: 0x74f7 [validation disabled]
[Header checksum status: Unverified]
Source: 192.168.1.100
Destination: 192.168.1.20
User Datagram Protocol, Src Port: 47998, Dst Port: 62455
Source Port: 47998
Destination Port: 62455
Length: 1416
[Checksum: [missing]]
[Checksum Status: Not present]
[Stream index: 4]
Without the full context of the packet capture, I can't tell whether that packet is expected or not. When Moonlight is disconnecting, we can expect to sometimes see a port unreachable message from the client to the server since the client has closed its socket but the server doesn't know yet.
I'm also curious about those TLS packets you mentioned and what ports those were on?
Can you send me a link to the full pcap file at [email protected]?
Please try disabling location services. It might interrupt your stream when it scans for nearby wifi networks.
System Preferences -> Security & Privacy -> Location Services
@starchaser01 🏆 🎖 👏 👏
That was it. I haven't had any issue since turning off. I will test further!
No issues at all. The location services caused the hiccups for me. Thanks all!
This had been driving me crazy for a couple of months since using a Mac until I found this issue. It only affects WiFi.
@cgutman it would be awesome if there was a small message that appears on macOS that notified you about this problem (like first time you start Moonlight), I can open another issue if you'd like.
I think there is now a new variant of this issue... I know it sounds insane, but is it possible for the iPhone 11 to interfere with the connection when using Moonlight on MacOS? Every time I wake up an iPhone 11 in proximity to my laptop, regardless on if it is on the WiFi network/bluetooth, Moonlight stutters for 10+ seconds afterwards. This time, disabling location services results in no improvement. Can somebody else replicate this setup?
Host Computer: Windows 10 custom built w/ ethernet Streaming Device: MacBook Pro 2015, macOS Catalina, AC Wifi Phone: iPhone 11 Pro Max
Is it possible the new U1 / wideband chip triggers some weird signal interferes with moonlight? This issue did not exist with the iPhone XS. Likewise, this issue does not affect Steam In Home Streaming. How does this make any sense?
I have the same issue as @eaferrari24 Host Computer: Windows 10 w/ ethernet Streaming Device: MBP 2019, Catalina, Ethernet Phone: iPhone X
This also occurs, when my Apple Watch screen turns on. Maybe we should open a new issue.
I think there is now a new variant of this issue... I know it sounds insane, but is it possible for the iPhone 11 to interfere with the connection when using Moonlight on MacOS? [...]
I think I used to have this, and I suspect it was due to Continuity/Handoff/etc. Maybe try disabling Handoff? I'm not sure if this is just in System Preferences or elsewhere. Maybe also check that you have AirDrop turned off.
Also, I'm guessing this doesn't happen if you use wired ethernet and turn off the wifi?
My vague recollection is that Continuity/Handoff works via Apple Wireless Direct Link (AWDL), which technically involves the wifi radio switching between channels rapidly. Perhaps see also https://stackoverflow.com/a/51786787 for further details.
@dsedivec Thank you for the insight and sharing the link. I’ll try disabling Handoff/AirDrop but having those features enabled has never been a Moonlight problem for me on past MacOS/iOS/iPhones. Likewise, this mystery doesn’t affect Steam in-home streaming.
The other reason I don’t think it is handoff related is because other iPhone 11’s not affiliated with my Apple ID trigger this jitter/packet loss on my moonlight game stream when in proximity to my laptop.
Honestly, I haven’t really followed up on this in a while. I mostly just keep my iPhone on the other side of the house or off when playing games and rely on MacOS to notify me of iMessage/FaceTime/Calls.
The post above you by @Cludch suggests it also happens when connected via Ethernet. The iPhone X doesn’t have the U1 chip, so I’m stumped. Perhaps it is yet another variant of this problem. I’ve personally never had the iPhone X/XS trigger this jitter/packet loss just from waking the screen.
My memory is vague but I recall replicating this issue in airplane mode... which makes me think the U1 chip doesn’t obeys the iOS airplane mode setting. Apple did recently confirm that the currently released versions of iOS continue to broadcast the U1 even if location services are disabled.
Considering this doesn’t affect moonlight streaming onto the Apple TV or iPhone 11 and Steam in-home streaming works fine in all settings, I don’t think it’s the shield protocol or a macOS networking driver/interface issue. It has to be something with the macOS implementation of moonlight. Isn’t Steam’s in-home streaming open source? Maybe they communicate with the networking hardware in a different way? I’m out of ideas.
@dsedivec [...] Likewise, this mystery doesn’t affect Steam in-home streaming.
If that's the case, I would like to shift my suspicion to Qt. Quick story: I used to use a VoIP client written in Qt, and I would experience a huge ping spikes during calls on a regular basis. It turns out, Qt makes macOS rescan wireless networks periodically, which has the same effect as having location services on. I don't think this is related to your problem, since yours happens only when you wake up your phone, but I think it demonstrates that Qt can do some surprising things to the system.
Still, I doubt Qt on macOS has anything to do specifically with iPhones, so it would have to be something like "Qt causes a delay when a new BT/wifi device appears, which only happens when the iPhone wakes up". Something like that.
I assume you're on wifi and not wired ethernet. If so, maybe you could enable wifi logging (https://apple.stackexchange.com/a/348574) and see if you can spot any hints as to what happens when you wake your iPhone?
The other reason I don’t think it is handoff related is because other iPhone 11’s not affiliated with my Apple ID trigger this jitter/packet loss on my moonlight game stream when in proximity to my laptop.
That is weird. iPhone is not connected via USB? Is the phone paired to your computer with Bluetooth at all?
Honestly, I haven’t really followed up on this in a while. I mostly just keep my iPhone on the other side of the house or off when playing games and rely on MacOS to notify me of iMessage/FaceTime/Calls.
Understood. I just happened upon this issue after finding that I, too, needed to disable location services for a smooth experience (and sometimes quit/restart Firefox afterwards). Feel free to ignore my comments, thought I'd throw some ideas out. I just found it terribly disappointing (and, if you'll permit me, kind of funny) that you have to put your phone in another room to play games.
The post above you by @Cludch suggests it also happens when connected via Ethernet. The iPhone X doesn’t have the U1 chip, so I’m stumped. Perhaps it is yet another variant of this problem. I’ve personally never had the iPhone X/XS trigger this hitter/packet loss just from waking the screen.
My memory is vague but I recall replicating this issue in airplane mode... which makes me think the U1 chip doesn’t obeys the iOS airplane mode setting. Apple did recently confirm that the currently released versions of iOS continue to broadcast the U1 even if location services are disabled.
Since AFAIK no MacBooks have U1 chips, the only way I can think of the U1 chip playing a role here is if you're getting your network through wifi and the UWB signals are interfering with wifi. If that's the case, and it's not just a defect in your particular phone, I suspect the FCC (or your national equivalent) and Apple would very much like to hear about this!
Does your iPhone leave Bluetooth on in airplane mode? I noticed mine does, presumably since newer iPhones need BT for the headphone jack.
At some point in troubleshooting this I would have to start wondering if there is a defect with your specific iPhone and it's giving off RF interference.
[...] It has to be something with the macOS implementation of moonlight. Isn’t Steam’s in-home streaming open source? Maybe they communicate with the networking hardware in a different way? I’m out of ideas.
I don't know if it's open source, but I wouldn't be surprised if moonlight-qt is using Qt's networking facilities, whereas Steam in-home streaming is not using Qt for networking (if it's using Qt at all).
Still, I doubt Qt on macOS has anything to do specifically with iPhones, so it would have to be something like "Qt causes a delay when a new BT/wifi device appears, which only happens when the iPhone wakes up". Something like that.
I think you and I are in sync on our suspicions with QT.
At this point, I've seen this happen from multiple iPhone 11's so I really don't think its a defective phone. Likewise, since it doesn't affect Steam In Home Streaming I don't think it is appropriate yet to get Apple or the FCC involved. Yes, one iPhone has never been connected to my MacBook in any way, wired or wireless, and causes the issue. My airplane mode does leave bluetooth on, but I also manually toggled everything off.
This variant of the issue REALLY messes up the connection compared to the original location services packet loss this issue was opened for. I'll try to get some wifi logging info in the next few weeks.
Valves Open Source GitHub with a fair amount of their projects: https://github.com/ValveSoftware
Since the issue was re-opened, maybe someone will have a similar configuration that they can replicate! Thank you @dsedivec & @cgutman
Below is all speculation, but has anybody done any investigation into the UWB frequency range? Maybe it isn't too far off from 5ghz and is an independent chip purely for efficiency reasons. This may be why the newest 16 inch MacBook released a month ago doesn't have a dedicated chip.
EDIT: https://www.techinsights.com/blog/apple-u1-delayering-chip-and-its-possibilities
Moonlight does use Qt for networking, but only the HTTPS part required to pair and start a stream. Once the stream is going, we use the BSD sockets interface directly.
I don't think we're affected by QTBUG-40332 since we've always used Qt 5.11 or later (now Qt 5.14) which doesn't have the pinger that results in the inconsistent ping latency. WiFi logging would be useful along with Bluetooth logging if that's possible too. Along those lines, has anyone tried disabling Bluetooth on the Mac to see if that changes things?
Steam IHS is not open-source. It's provided as a binary blob for all platforms, so we can't directly study the source code.
Okay. I've had some time to play around with Moonlight more and I'm fairly certain the vast majority of my connection drop outs are related to AirDrop as @dsedivec suspected. This also explains why an iPhone 11 not connected to my account or network would still trigger the streaming issues (everyone/contacts only settings). As soon as I switched my AirDrop on my phone to "Contacts Only" and my AirDrop on my laptop set to not discoverable, the issue has seemingly disappeared. AirDrop discovery seems to check on a more frequent basis on the iPhone 11 than older iPhones. Having any app on a phone that may be a use case for AirDrop will trigger regular AirDrop discoverability scans which are noted in WiFi Logging. Any time a phone opens the share window on an iOS device triggers a discoverability scan unless AirDrop discoverability is disabled on macOS.
<kernel> AWDL Sync Enabled 1, data path state[1]
<kernel> AWDL Sync Enabled 0, data path state[1]
As far as Steam In Home streaming goes. AirDrop does affect it as well, but because of Steams dynamic quality/resolution feature it was less jarring and apparent.
I'm not sure why AirDrop would also be messing with an ethernet connection as I am suspecting based on the report by @Cludch nor do I understand why airdrop would be running on an Apple Watch. I don't think my watch has caused any moonlight stutters. I will be buying a thunderbolt ethernet adapter for further testing.
@eaferrari24 I might have been mistaken. The issue does not seem to appear with an ethernet connection. I thought I was using an ethernet connection, but I wasn't.
This issue still persists even after disabling location services and bluetooth.
I am experience this issue as well, however it's not happening after a few minutes but literally every 5-10 seconds.
The stream itself is perfect and does not hitch at all. But the mouse input has significant lag and will skip frames and stutter. I will notice the mouse stutter while images moving in the background move perfectly smooth.
Host: Windows 10 Home 1903 with 2080TI and i6-6600k Client: Mid 2015 15" Macbook Pro on 10.14.6
I've honestly tried everything. Lowered the stream all the way to 720p at 30fps with 5 mbps bitrate.
Toggled pretty much every setting as well and nothing works.
