fivem icon indicating copy to clipboard operation
fivem copied to clipboard

feat(state)!: add compression dictionary for RedM

Open AvarianKnight opened this issue 8 months ago • 2 comments

Goal of this PR

Having a compression dictionary for RedM reduces the net traffic from 18-30kb/s to 16-24kb/s in testing, depending on the amount of players in the area this could result in a lot less data sent over svNetwork

This is protocol breaking since our compression dict will no longer be the same one used for FiveM, this adds a protocol version file that can be shared between server/client to the bump easier if its ever done again

This PR applies to the following area(s)

RedM

Successfully tested on

Game builds: ..

Platforms: Windows, Linux

Checklist

  • [x] Code compiles and has been tested successfully.
  • [x] Code explains itself well and/or is documented.
  • [x] My commit message explains what the changes do and what they are for.
  • [x] No extra compilation warnings are added by these changes.

AvarianKnight avatar Apr 29 '25 19:04 AvarianKnight

For an explanation on how this was generated:

There's a function in CloneManagerLocal::SendUpdates to write chunks to a file, you can enable by setting #define WRITE_BYTEHUNK true (you'll have to modify the relative path to data/cache since this was used before the cache file was moved)

Afterwards just play in the game for a little bit and it'll write all the update packets it sends out and you can use zstd --train -r dirname outfile --maxdict=64KB to generate the raw dictionary file

After that you can use this python script to convert the dictionary file to a header file of bytes

convert_dict_to_header.zip

AvarianKnight avatar Apr 30 '25 15:04 AvarianKnight

It should be noted that it would be best to merge this with the object id patch when it's finished (and train it on that patch too) since it is also protocol breaking, it would be better to break all at once than to do two separate breakages.

AvarianKnight avatar Apr 30 '25 15:04 AvarianKnight