Unexpected logout caused by network reconnection

[cdwcgt]

Type

Game behaviour

Bug description

SpectatorClient may disconnect because network problem, after re-connect, user will disconnect due to You have been logged out on this device due to a login to your account on another device.

Screenshots or videos

No response

Version

2023.1231.0

Logs

compressed-logs.zip

check 1704173063.***.log

need to attention:

runtime.log

2024-01-02 11:58:54 [verbose]: ⚠️ You have been logged out on this device due to a login to your account on another device.

network.log

2024-01-02 11:58:46 [verbose]: SpectatorClient connect attempt failed: Server timeout (30000.00ms) elapsed without receiving a message from the server.
2024-01-02 11:58:51 [verbose]: SpectatorClient connecting...
2024-01-02 11:58:54 [verbose]: SpectatorClient connected!

[bdach]

Seems to be caused by partial disconnection from osu-server-spectator (only one hub out of three). Not sure how to fix (and probably low prio anyway due to low report volume).

[IsleNauts]

same issue here.

Logs: compressed-logs.zip

[Stehsaer]

Experiencing same problem here with version 2024.521.2. Happening around once every 2~3hrs. Very experience-breaking.

[bdach]

I found one way to reproduce, namely apply the following patch:

diff --git a/osu.Game/Online/API/APIAccess.cs b/osu.Game/Online/API/APIAccess.cs
index 923f841bd8..630d8db828 100644
--- a/osu.Game/Online/API/APIAccess.cs
+++ b/osu.Game/Online/API/APIAccess.cs
@@ -78,6 +78,11 @@ public partial class APIAccess : Component, IAPIProvider
 
         protected bool HasLogin => authentication.Token.Value != null || (!string.IsNullOrEmpty(ProvidedUsername) && !string.IsNullOrEmpty(password));
 
+        public void RotateToken()
+        {
+            authentication.AuthenticateWithRefresh(authentication.Token.Value.RefreshToken);
+        }
+
         private readonly CancellationTokenSource cancellationToken = new CancellationTokenSource();
 
         private readonly Logger log;
diff --git a/osu.Game/Online/Spectator/OnlineSpectatorClient.cs b/osu.Game/Online/Spectator/OnlineSpectatorClient.cs
index 036cfa1d76..eeb190c522 100644
--- a/osu.Game/Online/Spectator/OnlineSpectatorClient.cs
+++ b/osu.Game/Online/Spectator/OnlineSpectatorClient.cs
@@ -3,6 +3,7 @@
 
 using System;
 using System.Diagnostics;
+using System.Reflection;
 using System.Threading.Tasks;
 using Microsoft.AspNetCore.SignalR.Client;
 using osu.Framework.Allocation;
@@ -124,5 +125,13 @@ protected override async Task DisconnectInternal()
 
             await connector.Disconnect().ConfigureAwait(false);
         }
+
+        public void FakeServerTimeout()
+        {
+            // time for some SHENANIGANS
+            var method = typeof(HubConnection).GetMethod("OnServerTimeout", BindingFlags.NonPublic | BindingFlags.Instance);
+            if (connection != null)
+                method!.Invoke(connection, []);
+        }
     }
 }
diff --git a/osu.Game/OsuGameBase.cs b/osu.Game/OsuGameBase.cs
index 5e4ec5a61d..3b90490d05 100644
--- a/osu.Game/OsuGameBase.cs
+++ b/osu.Game/OsuGameBase.cs
@@ -21,6 +21,7 @@
 using osu.Framework.Graphics.Containers;
 using osu.Framework.Graphics.Textures;
 using osu.Framework.Input;
+using osu.Framework.Input.Events;
 using osu.Framework.Input.Handlers;
 using osu.Framework.Input.Handlers.Joystick;
 using osu.Framework.Input.Handlers.Midi;
@@ -61,6 +62,7 @@
 using osu.Game.Scoring;
 using osu.Game.Skinning;
 using osu.Game.Utils;
+using osuTK.Input;
 using RuntimeInfo = osu.Framework.RuntimeInfo;
 
 namespace osu.Game
@@ -715,5 +717,22 @@ protected override void Dispose(bool isDisposing)
         ControlPointInfo IBeatSyncProvider.ControlPoints => Beatmap.Value.BeatmapLoaded ? Beatmap.Value.Beatmap.ControlPointInfo : null;
         IClock IBeatSyncProvider.Clock => beatmapClock;
         ChannelAmplitudes IHasAmplitudes.CurrentAmplitudes => Beatmap.Value.TrackLoaded ? Beatmap.Value.Track.CurrentAmplitudes : ChannelAmplitudes.Empty;
+
+        protected override bool OnKeyDown(KeyDownEvent e)
+        {
+            if (e.Key == Key.PageUp && !e.Repeat)
+            {
+                ((APIAccess)API).RotateToken();
+                return true;
+            }
+
+            if (e.Key == Key.PageDown && !e.Repeat)
+            {
+                ((OnlineSpectatorClient)SpectatorClient).FakeServerTimeout();
+                return true;
+            }
+
+            return base.OnKeyDown(e);
+        }
     }
 }

and then press page up to refresh the local oauth token and then page down to fake the timeout.

Cause is the oauth token changing while the game is running, but the underlying hub connectors not being aware of it until the next connection blip, at which point the token change is interpreted as another separate client connecting.

Not sure what the fix is yet. Possible directions include using something other than API tokens (that is ideally not trivially spoofable) or adding a way to notify the server that the client has rotated its token.

I'm somewhat skeptical this will be the end of it because it seems like a long shot that certain users would experience this regularly unless they leave their game open for long hours overnight or something.

[bdach]

I did a change today that was intending on addressing this by adding a primitive to let the server know that the client's token has changed, but I'm honestly not convinced by the complexity of the final result:

https://github.com/bdach/osu/tree/token-switch-handling https://github.com/bdach/osu-server-spectator/tree/token-switch-handling

The move here is to probably move away from using JWTs for session identification and use a client-side generated guid instead. Will try this next week.

[bdach]

Reopening for a while longer, github's keyword match was a bit too eager there (still need the server side change for a full fix).

[peppy]

Server-side change has been merged and deployed.