Efficiently syncing data between the server and client (or vice versa)

[SABogdan]

Goal

Efficient synchronization of data between the server and client (or vice versa). This data CAN include dictionaries, lists, and classes. The current process of synchronization is inefficient, cumbersome and not optimal as it is time-consuming and can lead to performance issues, especially when dealing with large datasets as it involves converting these complex data structures into JSON strings and then parsing them back into their original form. Example:

//server
public Dictionary<int, someClass> _serverDictonary = new Dictionary<int, someClass>();
public void serverFunction()
{
..some logic
TriggerClientEvent("someEvent", Json.Stringify(_serverDictonary));
}

//client
public Dictionary<int, someClass> _clientDictonary = new Dictionary<int, someClass>();
[EventHandler("someEvent")]
public void OnSync(string json) 
{
 _clientDictonary = Json.Parse<Dictionary<int, someClass>>(json);
..some logic
}

Importancy

Quality of Life (QoL)

API and/or potential implementation

// Server
public Dictionary<int, someClass> _serverDictionary = new Dictionary<int, someClass>();

public void ServerFunction()
{
    // ... some logic ...
    
    TriggerClientEvent("someEvent", _serverDictionary);
}

// Client
public Dictionary<int, someClass> _clientDictionary = new Dictionary<int, someClass>();

[EventHandler("someEvent")]
public void OnSync(Dictionary<int, someClass> serverData) 
{
    _clientDictionary = serverData;
    
    // ... some logic ...
}

Extra

No response

[Local9]

Your implementation should still work without really any changes as MsgPack is used to send data back and forth between the client and the server, but I would also add that your implementation is rather 'cumbersome' too, I would like to see events become more of an RPC design so I don't have to even use the Trigger* Natives.

Recommend as a working example https://github.com/manups4e/FxEvents

Currently has a beta NuGet for Mono v2; any issues let us know.

https://github.com/manups4e/FxEvents/pull/13

https://www.nuget.org/packages/FxEvents.FiveM.Client/2.0.0-beta3-monov2 https://www.nuget.org/packages/FxEvents.Server/2.0.0-beta3-monov2

To show an example using yours but with FxEvents;

// Server
public class MyServer : BaseScript
{
	public Dictionary<int, someClass> _serverDictionary = new Dictionary<int, someClass>();
	
	public MyServer()
	{
		EventDispatcher.Mount("eventName", new Action<ISource>(([FromSource] source) =>    
		{
		  return _serverDictionary;
		}));
	}
}

// Client
public Dictionary<int, someClass> _clientDictionary = new Dictionary<int, someClass>();
public void OnSync() 
{
	 _clientDictionary = await EventDispatcher.Get<Dictionary<int, someClass>>("eventName", params);
	// ... some logic ...
}

[thorium-cfx]

The current process of synchronization is inefficient, cumbersome and not optimal as it is time-consuming and can lead to performance issues

Although I agree with it being cumbersome, you may want to look into Network Engineering before you make claims about the others. Your concern should be around data packets when you're designing systems around networking; as the more data you send the slower it gets (especially due to packet loss). Naïvely serializing classes produces a lot of unnecessary data, so setting up your own protocol will (if done correctly) always outperform the naïve approach (including JSON).

This is indeed a time-consuming process, but for performance and efficiency a custom protocol is always required.

Continuing on that knowledge, implementing something like this would need a design in which serialization and deserialization can be manually done, to reduce and pack data, but also to not use reflection (bye bye performance). This'll be what most time needs to be spend on.

[github-actions[bot]]

This issue is marked as stale because it has been open for 30 days without any activity.

[thorium-cfx]

This issue will be solved with the upcoming MsgPack serializer update, allowing serialization and deserialization from and to any type, with customizability support, e.g.: map based and array based type (de)seriailzation, ignore and include fields, etc.

[manups4e]

Finally.. FxEvents will be obsolete!! ❤️

[IstuntmanI]

The changes proposed by that PR were nice, too bad that they have been meanwhile reverted due to various issues.

Any news about it, @thorium-cfx? Is it still a priority?

[DaniGP17]

The changes proposed by that PR were nice, too bad that they have been meanwhile reverted due to various issues.

Any news about it, @thorium-cfx? Is it still a priority?

Thorium leave CFX and mono is being supported by PRs