[FEATURE] Dedicated faster serialization when shuffle writing/reading

[zuston]

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

Search before asking

• [x] I have searched in the issues and found no similar issues.

Describe the feature

After overlapping compression is applied, the majority of shuffle write times is the seralization. Is it possible to replace the kyro serialization with the fast implementation like apache fory? It looks it has a impressive benchmark result .

https://github.com/apache/fory/blob/main/docs/benchmarks/serialization/bench_serialize_STRUCT2_to_array_tps.png

Motivation

No response

Describe the solution

No response

Additional context

No response

Are you willing to submit PR?

• [x] Yes I am willing to submit a PR!

[zuston]

cc @jerqi

[jerqi]

cc @jerqi

There are some points that:

1. The type system of serialization
2. supportsRelocationOfSerializedObjects

[jerqi]

cc @zhengchenyu

[zuston]

cc @jerqi

There are some points that:

1. The type system of serialization
2. supportsRelocationOfSerializedObjects

Yes. I will draft a simple impl to benchmark