citizenfx
feat(scripting/v8): expose msgpack codec globally for custom type sup…
Goal of this PR
Expose the msgpack codec globally to enable developers to easily add custom serializers and deserializers, enhancing flexibility for handling custom types in FiveM scripts.
How is this PR achieving the goal
This PR adds a new global variable, global.msgpack_codec, which provides direct access to the existing msgpack codec. Developers can use this to register custom types via addExtPacker and addExtUnpacker while maintaining backward compatibility with the current global.msgpack_pack and global.msgpack_unpack functions.
This PR applies to the following area(s)
- Core scripting framework
- Data serialization/deserialization using msgpack
Successfully tested on
Game builds:: 3258
Platforms: Windows
Checklist
- [x] Code compiles and has been tested successfully.
- [x] Code explains itself well and/or is documented.
- [x] My commit message explains what the changes do and what they are for.
- [x] No extra compilation warnings are added by these changes.
Fixes issues
No specific issues reported for this feature addition.
Here is a link to my documentation pull request: https://github.com/citizenfx/fivem-docs/pull/503
Personally I'd like to see a modern and more efficient msgpack implemented before we get access to the codec.
The current one hasn't been touched for 8 years and is lacking a lot of JS features. https://github.com/kawanet/msgpack-lite
@AvarianKnight might also have some thoughts on this.
Personally I'd like to see a modern and more efficient msgpack implemented before we get access to the codec.
The current one hasn't been touched for 8 years and is lacking a lot of JS features. https://github.com/kawanet/msgpack-lite
@AvarianKnight might also have some thoughts on this.
There's a lot better encoders/decoders for msgpack now and it would make a lot of sense to swap to them (see msgpackr for one that's really fast, and blows out all current implementations)
Personally I'd like to see a modern and more efficient msgpack implemented before we get access to the codec. The current one hasn't been touched for 8 years and is lacking a lot of JS features. https://github.com/kawanet/msgpack-lite @AvarianKnight might also have some thoughts on this.
There's a lot better encoders/decoders for msgpack now and it would make a lot of sense to swap to them (see msgpackr for one that's really fast, and blows out all current implementations)
Thank you for the suggestion! (I don't know if this message is directed to the right place.) I’ve been researching and testing msgpackr, and I agree it’s an impressive library with great performance. However, implementing it poses some significant challenges.
Here’s what I’ve managed so far and the issues I’ve encountered:
- Progress Achieved:
- I’ve successfully implemented msgpackr for standard arguments serialization/deserilization.
- Function References work as excepted when triggering from LUA to JS with a LUA function as argument.
- Challenges Remaining:
- Function References: I haven’t been able to make function references work correctly when triggering from JS to LUA.
Note: I have not made any test with C#.
These challenges would require additional development to resolve and could impact other parts of the project. I’d love to hear your thoughts or suggestions on addressing these issues.
Additionally, if you'd like, I can share the code I’ve worked on so far to help provide more context. Please let me know where and how I should share it.
Hello again 👋
Now that the migration to msgpackr (PR #3201) was reverted, I’d like to revisit the original intent of this PR.
The goal here was not to replace the codec, but to expose the existing one globally, so developers can register custom serializers/deserializers using addExtPacker / addExtUnpacker.
This would provide more flexibility for advanced scripting scenarios while maintaining full backward compatibility.
Since the codec (msgpack-lite) is still in place and actively used, could we reconsider merging this lightweight exposure?
I'm happy to rebase and update the PR if needed.
@Nobelium-cfx – would love to hear your thoughts on this. 🙂
@Nobelium-cfx – would love to hear your thoughts on this. 🙂
I dont really know much about script runtimes and msgpack . So forgive me if I say something stupid.
I understand the general argument for flexibility. But, to be honest, I have a hard time coming up with a scenario where this would be actually useful. Can you provide a particular example of a feature that can't be implemented without this (or is significantly harder to implement)?
Another concern is: do we only execute addExtPacker / addExtUnpacker inside sandbox? Because if not - I believe this change may expose a vulnerability and allow server to damage client machines.
@Nobelium-cfx – would love to hear your thoughts on this. 🙂
I dont really know much about script runtimes and
msgpack. So forgive me if I say something stupid.I understand the general argument for flexibility. But, to be honest, I have a hard time coming up with a scenario where this would be actually useful. Can you provide a particular example of a feature that can't be implemented without this (or is significantly harder to implement)?
Another concern is: do we only execute
addExtPacker/addExtUnpackerinside sandbox? Because if not - I believe this change may expose a vulnerability and allow server to damage client machines.
One practical use-case is sending a large array of objects that all share the same properties in an event. With addExtPacker/addExtUnpacker exposed, we could register a custom extension to create a more compact payload, which was one of the main wins we saw with msgpackr over msgpack-lite.
Right now the only way to control packing at that level is to use TriggerServerEventInternal on the sender and global.addRawEventListener on the receiver, to bypass msgpack-lite serialisation. It's still possible, but involve using 'internal' functions
@Nobelium-cfx – would love to hear your thoughts on this. 🙂
I dont really know much about script runtimes and
msgpack. So forgive me if I say something stupid.I understand the general argument for flexibility. But, to be honest, I have a hard time coming up with a scenario where this would be actually useful. Can you provide a particular example of a feature that can't be implemented without this (or is significantly harder to implement)?
Another concern is: do we only execute
addExtPacker/addExtUnpackerinside sandbox? Because if not - I believe this change may expose a vulnerability and allow server to damage client machines.
Thanks for the feedback, @Nobelium-cfx 🙏
Let me try to clarify a few points that might help address your concerns:
✅ No New Vulnerability Introduced
This PR does not introduce any new capability that wasn’t already available internally.
The msgpack-lite codec is already used behind the scenes for msgpack_pack and msgpack_unpack, as well as for addEventListener, on, onNet, TriggerEvent, TriggerServerEvent, and TriggerClientEvent.
This change simply exposes the codec via a global msgpack_codec variable, within the same JS sandbox environment scripts already run in.
Developers can then register custom types using addExtPacker / addExtUnpacker, which are existing features of the library.
There is no way for this to break out of the sandbox or gain additional permissions. It’s just surfacing an internal object for extensibility.
🔐 Still Fully Sandboxed
All usage remains:
- Within the existing JS runtime sandbox.
- Subject to the same runtime permission boundaries as current scripting features.
So from a security standpoint, this doesn’t open any new attack surface—it simply enables advanced use cases for developers already operating within safe boundaries.
🎯 Real-World Use Case
Imagine a developer needs to serialize a custom class like Money, Vector3, or Color. Currently, they need to manually encode/decode data using lower-level calls like this:
const codec = msgpack.createCodec({
preset: true,
binarraybuffer: true,
uint8array: true
});
class Money {
constructor(public amount: number, public currency: string) {}
}
codec.addExtPacker(0x42, Money, (money) => [money.amount, money.currency]);
codec.addExtUnpacker(0x42, ([amount, currency]) => new Money(amount, currency));
addEventListener('money', (rawMoney) => {
const [amount, currency] = msgpack.decode(rawMoney, {codec});
const money = new Money(amount, currency);
console.log(money, money instanceof Money); // Output { amount : 10, currency: '$' }, true
});
const money = new Money(10, '$');
TriggerEvent('money', msgpack.encode(money, {codec}));
This works, but adds overhead and repeated boilerplate.
By exposing msgpack_codec, the same behavior becomes more declarative and reusable:
class Money {
constructor(public amount: number, public currency: string) {}
}
msgpack_codec.addExtPacker(0x42, Money, (money) => [money.amount, money.currency]);
msgpack_codec.addExtUnpacker(0x42, ([amount, currency]) => new Money(amount, currency));
addEventListener('money', (money) => {
// No more msgpack.decode call here
console.log(money, money instanceof Money); // Output { amount : 10, currency: '$' }, true
});
TriggerEvent('money', new Money(10, '$')); // No more msgpack.encode call here
This not only reduces complexity and error-prone conversions, but also enables cleaner architecture when sharing complex types across JS ↔ Lua boundaries.
I'm totally open to suggestions—whether it’s renaming the global or limiting visibility to certain scopes—to make this more comfortable for review. But I do think this provides tangible value to developers without compromising safety.
In short, this change just gives devs access to a tool they already rely on indirectly—now with full control and clarity.
Looking forward to your thoughts! 🙂
What is with people and using AI for GitHub issues and PRs? It's painfully obvious that it's AI, it does nothing but take away from what you are trying to say
What is with people and using AI for GitHub issues and PRs? It's painfully obvious that it's AI, it does nothing but take away from what you are trying to say
He doesn't know a lot about msgpack so i explain it to him but i'm not english so i use IA to make my comment better. I make examples, and they are clear to understand, what more do you expect ?
He doesn't know a lot about msgpack so i explain it to him but i'm not english so i use IA to make my comment better. I make examples, and they are clear to understand, what more do you expect ?
if he wanted an AI response he couldve done it himself, theres no way of knowing if this is your words or some randomly generated non sense
While its AI generated, its all (mostly) correct. You wouldn't be able to decode the type without adding to the codec, since you don't have access to the codec.
There isn't any issues relating to sandboxing, since extending the msgpack codec would be per-ScRT, Lua already has the capability to do this.
Can you provide a particular example of a feature that can't be implemented without this (or is significantly harder to implement)?
Sending Vectors out of the JS runtime wont encode properly to a Lua version, this makes cross compat between the two annoying (at least in regards to sending vectors, though someone figured out an extremely hacky work around to this to get the codec, but I don't remember what it was)
This also isn't anything new, Lua already exposes its codec.
While its AI generated, its all correct.
There isn't any issues relating to sandboxing, since extending the msgpack codec would be per-ScRT, Lua already has the capability to do this.
Can you provide a particular example of a feature that can't be implemented without this (or is significantly harder to implement)?
Sending Vectors out of the JS runtime wont encode properly to a Lua version, this makes cross compat between the two annoying (at least in regards to sending vectors, though someone figured out an extremely hacky work around to this to get the codec, but I don't remember what it was)
This also isn't anything new, Lua already exposes its codec.
In my case it's to make instance replicated in both client/server in JS env using this codec
So i can create Ped instance server side and get it back in client side for example. I have already done this in lua and it's time saver