orleans icon indicating copy to clipboard operation
orleans copied to clipboard

Support MemoryPack serialization

Open jerviscui opened this issue 4 months ago • 9 comments

I want to extend serialization using MemoryPack.

I added Orleans.Serialization.MemoryPack to the Orleans repository, but now my tests are failing.

I noticed the MemoryPack Source Generator isn't running during compilation. Why?

Can you help me? @ReubenBond

jerviscui avatar Sep 07 '25 03:09 jerviscui

The code at https://github.com/jerviscui/orleans/blob/memorypack/test/Orleans.Serialization.UnitTests/MemoryPackSerializerTests.cs

jerviscui avatar Sep 07 '25 14:09 jerviscui

@neuecc Help 🙋‍♂️

jerviscui avatar Sep 07 '25 17:09 jerviscui

https://github.com/dotnet/orleans/issues/9644

jerviscui avatar Sep 07 '25 17:09 jerviscui

Dude, implement your memory pack serialization wrapper and add it via .AddSerialization 🤨

0xF6 avatar Sep 08 '25 15:09 0xF6

Dude, implement your memory pack serialization wrapper and add it via .AddSerialization

MemoryPack serialization works in my code, but the Orleans unit test fails.

jerviscui avatar Sep 09 '25 12:09 jerviscui

MemoryPack serialization works in my code, but the Orleans unit test fails.

You don't need to change serialization of the system types of orleans itself, memopack only makes sense for large amounts of user-specific data or streaming data for compatible with nats\etc

0xF6 avatar Sep 09 '25 12:09 0xF6

Hi @0xF6, Looking solely at serialization performance tests, MemoryPack is the fastest. MessagePackTest.ArraySerializeBenchmark-report-github.md

However, during Orleans request processing, I observed that Orleans.Serializer outperforms MemoryPackCodec. Test code: BenchmarkController.cs Run the API and SiloHost, then request the Controller.

Could you help analyze the cause? Is there an issue with the MemoryPackCodec implementation?

jerviscui avatar Sep 11 '25 03:09 jerviscui

@jerviscui please feel free to profile the benchmarks and determine the cause. It's not something I will likely look into soon. My initial hunch would be that there is an inefficiency in the delegation code, either because buffer copying is required or in looking up which serializer to delegate to. There is likely room for improvement there.

ReubenBond avatar Oct 02 '25 17:10 ReubenBond

@jerviscui please feel free to profile the benchmarks and determine the cause. It's not something I will likely look into soon. My initial hunch would be that there is an inefficiency in the delegation code, either because buffer copying is required or in looking up which serializer to delegate to. There is likely room for improvement there.

OK, when I have time, I will continue to explore.

jerviscui avatar Oct 29 '25 16:10 jerviscui