Usage of DateTime.UtcNow caused issues on IOS builds

[karisigurd4]

Hey, thought I'd share since I spent an inordinate amount of time scratching my head over this one 😅

First wanted to say thanks for sharing this library! Riptide is absolutely fantastic.

So the heartbeat mechanism uses DateTime.UtcNow to check whether the timeout period has elapsed. I'm developing for cross-platform so it always behaved very nicely on my PC and Android devices, however when it came to IOS builds I ran into sporadic connection issues where players would lose their connections.

I'm not sure exactly how DateTime works on IOS devices, for my "Round timer" component I have to do (TimeSpawn.FromMinutes(3) - timeLeft) for IOS wheras using only timeLeft seems to be equivalent on PC and Android. Another unfortunate point of frustration when it comes to the apple developer experience.

I solved it by simply removing the heartbeat mechanic entirely and handling it on the application side. But might be useful to take a look at within the library.

[tom-weiland]

First wanted to say thanks for sharing this library! Riptide is absolutely fantastic.

Glad you like it 🙂

for my "Round timer" component I have to do (TimeSpawn.FromMinutes(3) - timeLeft) for IOS wheras using only timeLeft seems to be equivalent on PC and Android.

Could you post some of your actual code (for iOS vs PC, including how you're calculating timeLeft)? Just based on what you've mentioned here, I can't even imagine what would have to be different/broken with DateTime for those to be equivalent...

I also did a quick google search but couldn't find anything related to strange DateTime behaviour on iOS devices 🤔

I solved it by simply removing the heartbeat mechanic entirely and handling it on the application side.

Keep in mind that Riptide's timeout system relies on heartbeats, so if you've removed heartbeats you may end up with connections "hanging" and not closing properly. For example, if the server shuts down and the disconnect message is lost in transmission (or the server crashes and just never has a chance to send said message), clients would normally time out after a few seconds because they've stopped receiving heartbeats. Obviously if you've removed heartbeats altogether, there will be no way for a server or client to tell if the other end has died, unless you've implemented an alternative time out system.

I'm not sure how involved it would be for you to revert your heartbeat removal change, but one thing you could try is increasing the server's & client's TimeoutTime (default is 5000 milliseconds). It wouldn't really be a proper solution, but whether or not doing so helps at all might provide some extra insight into what's going wrong.

At one point I was considering using a Stopwatch instead of DateTime to calculate the time since the last heartbeat, so I may have to revisit that if this does in fact turn out to be a problem with DateTime.

[karisigurd4]

Could you post some of your actual code (for iOS vs PC, including how you're calculating timeLeft)?

Yessir sorry for being so ambiguous was a bit of a scatterbrain yesterday. Here's an example of some of my DateTime adventures with IOS,

1. When my server starts up a RoundEndTime is defined as DateTime.UtcNow + TimeSpan.FromSeconds(180)
2. When a client joins a game one of the variables the server sends over is the RoundEndTime. I assign it to a global GameNetworkManager.Instance.RoundEndTime
3. For my in-game "time left" UI panel component I update the text indicator in the following way every second: GameNetworkManager.Instance.RoundEndTime - DateTime.UtcNow

Let's say 10 seconds have passed in an active game, on PC and Android, I see the following text: "2:50" however on the iPad I have to test with I see "57:10". I'm working with a group and they've got different variants of iPhones or iPads and according to the screenshots I've seen from them it's a bit hit and miss whether the timer is acting correctly. Seems to be very device dependant.

Definitely some weird Unity issue that they hopefully manage to fix at some point. I have too much on my plate atm to fully wrap my head around what's going on but I'm very scared of DateTime on IOS now. Since removing the heartbeat mechanism I haven't gotten any reports about suddenly losing connections etc., I have about 8 regular players testing with me (all running on IOS for some reason) and they've been super happy with the past few builds.

Keep in mind that Riptide's timeout system relies on heartbeats, so if you've removed heartbeats you may end up with connections "hanging" and not closing properly.

Indeed, there was also a secondary issue that when you move an application to the background, e.g., switching between apps on IOS the update logic in the game didn't run and that could lead to the heartbeat triggering a timeout once you came back. I've implemented a more lenient "AFK" mechanism in my server which allows connections to hang for the duration of a game and if they come back to the game I just respawn them and broadcast a notification that they've returned etc.

[tom-weiland]

It appears that my latest response was deleted/didn't post properly, which is kind of annoying since it was rather lengthy 😒

Let's say 10 seconds have passed in an active game, on PC and Android, I see the following text: "2:50"

I'm surprised the PC and Android times match—system times are not guaranteed to be identical, or even reasonably close together. I found this out the hard way a long time ago when I tested some client prediction code with a friend and it completely fell apart (after working perfectly on just my local machine). Eventually we realized that his computer's system time was ~30 seconds ahead of mine, so my server was ignoring his inputs because they were coming from ~30 seconds in the future.

Basically, DateTime objects can't really be safely compared to each other unless both were sampled on the same machine (or you don't need an overly precise comparison), so you may want to rethink how you're informing clients of how much time is left in a round.

However, this shouldn't cause any problems for Riptide, as I made sure to only compare DateTime timestamps when both were sampled on the same computer. A DateTime object sampled on the server will never be compared to one from the client, and vice versa.

It's worth noting that I've had a few other reports of seemingly random disconnections, most of which (possibly even all, I'm not entirely sure) were happening with mobile devices involved. Increasing the timeout time usually seems to "fix" it, but that's obviously not a proper solution. Unfortunately I've never had enough context/information to be able to pinpoint any potential causes, but it could just have something to do with how mobile devices handle multiple threads or something 🤔

Definitely some weird Unity issue that they hopefully manage to fix at some point.

Yeah, you may want to report it to Unity and/or Microsoft (given that DateTime is part of "regular" .NET it may not actually be Unity's fault, unless Unity somehow broke its implementation).

[tom-weiland]

1faf2a76617a3f3f339f5de8e4ca06838aa226e9 removes all use of DateTime from Riptide, so if that is in fact what was causing this issue, it should now be resolved.