fury
fury copied to clipboard
[Java] primitive array compression by SIMD
Is your feature request related to a problem? Please describe.
Currently primitive array are serialized by copy the data buffer directly using sun.misc.Unsafe
, this is super fast. Buf if all values in array are small, compression may reduce the data dramatically.
Compression needs to iterate the array at least once, which may be slow.
Describe the solution you'd like
- Using JDK 17 vector API for fast compression by SIMD.
- Providec an option to enable/disable this feature.
Additional context #108
Hello, can you describe the solution in more detail?
Hello, can you describe the solution in more detail?
Hi @heliang666s , thanks for the interest about this issue.
We may need to use SIMD api to detect whether all values in array are small value. If all values are small or first n values are small, we should use SIMD to cast it into smaller value into target buffer directly .
ok,I will try it.Please assign me thanks~