Serialize and deserialize sparse data structrure from/to buffer and file

[xuhao1]

Concisely describe the proposed feature I would like to serialize/deserialize sparse data structure with to a buffer or to file, this will benefit online transmission and data logging/replay

Describe the solution you'd like (if any)

#Serialize
B = B.pointer(ti.ijk, (K, K, K))
v = ti.field(ti.i32)
B.place(v)
#To buffer.
buf = ti.data_to_buffer(v)
#To file
ti.data_to_file(v)

#Deserialize
B, v = ti.from_buffer(buf) #B is the SNode, v is the value
B, v = ti.from_file(file)

Additional comments Personally, I think this feature is very important for using taichi for high-performance computation, which requires distributed computing and data logging. The reason the store/load sparse data is hope it can have compact structure to save the memory.

[jim19930609]

This seems to be a feature request to our AOT system. For now AOT only serializes the "NdarrayType" or "SNodeTreeType", and we should enable serialization for the underlying data buffers as well.

Updated LLVM AOT tasks to address this issue: https://github.com/taichi-dev/taichi/issues/4800

[xuhao1]

@jim19930609 Thanks! I will take a look.