Network variables out of sync for in scene network objects on clients

[TheCaveOfWonders]

If we have scene network objects that have network variables, when the clients load these objects by loading the scene (controlled by the Network Manager), these network variables will remain out of sync with the server IF the server modified these network variables WHILE the clients were still loading the scene.

Even though these scene objects were spawned on the server (in the network object component we can see the clients as observers) and the network variables had their values set on the server, the clients never get this update and their network variables will remain empty.

Workaround

The only way to trigger a resync with the server is to mark those network variables as dirty on the server OR to have the server write new values (different than the old values) into them.

Reproduce Steps

1. Have a scene with game objects (in my case they are prefabs) that have the network object component
2. start the server with a startup scene, different than that of step1.
3. connect the clients to the server, they will connect and load the startup scene.
4. have the server load the scene from step1 using the network manager, with the mode Additive.
5. as part of the OnNetworkSpawn of the game objects from step1, have the server write into their network variables. (so we are writing into them while clients are still loading the scene.)
6. observe that the connected clients will have their network variables from step1 be out of sync and remain empty.
7. on the server, observe that the network objects from step1 have all the clients as observers and the network variables all have values in them.

Actual Outcome

Network variables of the in scene network objects remain empty, they never got their values from the server.

Expected Outcome

Network variables of the in scene network objects should be in sync with the server.

Environment

• OS: Windows 10
• Unity Version: 2023.2.5
• Netcode Version: 1.7.1

[NoelStephensUnity]

@TheCaveOfWonders Are you still experiencing this issue and if so did you try updating to v1.8.1?

I just tried to replicate this issue and clients properly synchronize to the value set by the server. Regarding this:

IF the server modified these network variables WHILE the clients were still loading the scene.

This is the order of operations for scene loading:

1. Server-side: you start a Load scene event (in your case additively loaded)
2. Server-side: When the scene is loaded, the server locally spawns any in-scene placed NetworkObjects/prefabs (locally spawning means it does not send any spawn notificstion).
3. Server-side: Only after the scene has loaded and all newly instantiated in-scene placed NetworkObjects are locally spawned, the server sends the SceneEventMessage to clients that contains the scene to load, the loading mode, and then a block of data that that is the serialized locally spawned in-scene placed NetworkObjects.
4. Client-side: Upon receiving the SceneEventMessage, the client begins loading the scene in the specified loading mode and stores off the serialized NetworkObjects into a temporary buffer.
5. Client-side: Upon loading the scene, the client parses the serialized in-scene placed NetworkObjects and matches each entry with the local instances created during the loading of the scene.
6. Client-side: While deserializing, the in-scene placed NetworkObjects are locally spawned (but not before any NetVar data is already applied).

I used a simple script attached to the in-scene placed NetworkObjects placed in the scene to be additively loaded:

    public class NetVarSceneLoad : NetworkBehaviour
    {
        private NetworkVariable<int> m_SomeNetVar = new NetworkVariable<int>();


        public override void OnNetworkSpawn()
        {
            if (IsServer)
            {
                m_SomeNetVar.Value = 10;
            }
            else
            {
                Debug.Log($"Client-{NetworkManager.LocalClientId} NetVar = {m_SomeNetVar.Value}");
            }

            base.OnNetworkSpawn();
        }

    }

If the issue is not resolved and you think there are enough differences in the script you are using vs the above example, could you please provide that script so I can try replicating the issue?

[NoelStephensUnity]

@TheCaveOfWonders I was also wondering if you have tried increasing the NetworkManager.NetworkConfig.SpawnTimeout value to a higher value like 30+ seconds? This will defer messages received for NetworkObjects that are not yet spawned. If your project was started with an earlier version of NGO (i.e. 1.7 or lower) then it is set to the default of 1 second (the default now is 10 seconds but if the NetworkManager was added with an earlier version then that could still be 1 second).

[TheCaveOfWonders]

@TheCaveOfWonders I was also wondering if you have tried increasing the NetworkManager.NetworkConfig.SpawnTimeout value to a higher value like 30+ seconds? This will defer messages received for NetworkObjects that are not yet spawned. If your project was started with an earlier version of NGO (i.e. 1.7 or lower) then it is set to the default of 1 second (the default now is 10 seconds but if the NetworkManager was added with an earlier version then that could still be 1 second).

thanks for that hint, I didn't even know about NetworkManager.NetworkConfig.SpawnTimeout as it does not show up in the Inspector, and like you said since my NetworkManager was an old one, this was set to 1, I was able to see it by just opening my NetworkManager prefab in a text editor, I changed that to a 10.

[NoelStephensUnity]

@TheCaveOfWonders Just so I can verify, did adjusting that resolve your issue?

[TheCaveOfWonders]

@TheCaveOfWonders Just so I can verify, did adjusting that resolve your issue?

I'm unable to test it as I placed in a workaround, unfortunately it would be a bit of work now to remove that workaround in order to test this out.

That said it does seem like this was the culprit as my NetworkManager.NetworkConfig.SpawnTimeout was left at the default value of 1 second that came with the very early version of NGO, and clients loading the scene took way more than that to load.

thanks for you help :)

[MusicCorner]

Having same issue. So you're telling me that netcode won't autopmatically sync network variables and you need to handle it with NetworkConfig.SpawnTimeout = 30f;?? Doesn't sound to be right fix tbh... Why not just fix the issue itself and stop forcing devs to add workarounds to their code Unity?

@NoelStephensUnity that adjustment helped but it shouldn't be like this i believe. Any network variable should just work out of the box and the actual value set on the server side should be auto synced for all client on their start/network spawn despite the time they spend on loading to the scene/connecting to a server...

[NoelStephensUnity]

@MusicCorner I understand the frustration and confusion around this issue, and it only happens due to the earlier versions of NGO having a default spawn timeout value of 1 second (i.e. the value is serialized and applied from that point forward and to change this automatically would be problematic if it was intentionally set).

What could help explain this better is understanding what this value does. Basically, the spawn timeout value determines how long a client will defer (hold onto) a message targeting a NetworkBehaviour if the associated NetworkObject doesn't yet exist on the client side.

I have seen this issue occur the most when the server updates a NetworkVariable right after sending the synchronization information to a newly joined client but prior to receiving the SceneEventType.SynchronizeComplete message from the newly joined client (i.e. at that point the newly joined client will have synchronized completely so all NetworkObjects will have been spawned on the client side) and the SpawnTimeout is the older/legacy default value of 1 second.

This typically only shows up for projects that started with an earlier version of NGO where the spawn timeout value defaulted to 1 second (i.e. if it took 1.001 seconds to finished synchronizing it could drop any pending deferred messages). For projects that started with more recent versions of NGO where the default for NetworkConfig.SpawnTimeout is 10 seconds it typically is not an issue at all.

The update for this issue was in NGO v1.8.0, so my guess is that you had started with an earlier version of NGO (i.e. v1.7.1 or earlier) which still defaulted to the 1 second spawn timeout value and that part of the client connection process (relative to your project) updates one or more NetworkVariables just after the synchronization information had been sent to the client?

[NoelStephensUnity]

Closing this issue as it is most likely due to the spawn timeout setting.