Cytnx
Cytnx copied to clipboard
Support for pickle or other ways to save combined data, including tensors, to a file
In a typical application of tensor network algorithms, one runs the code for a certain set of parameters. Then, the results (and possilby intermediate results) shall be stored. Since such a run can involve hundred and thousands of tensors, one does not want to save each one in its own file. It would be better to have one file which contains all relevant data. In this way, only one or few files are created for every set of parameters. Since such runs are typically repeated for many different parameters, many files are created already. For each run, the number of files to take track of or share with colleges should therefore be reduced.
It would be great to have a feature which can save multiple tensors along with other data to a file. This data could, for example, be a tensor network state consisting of many tensors, together with other parameters like energies, observables, iteration numbers, (imaginary) time, runtimes etc.
Also, it would be great if this could be done in a standardized way, so not every developer has its own very specific file format and way to save data.
As we discussed before, Pythons pickle provides a way to store the data. One class containing all the data to be stored can be created and then saved and loaded with pickle. For this, pickle needs to be supported by the classes in the library, especially for tensors.
In the past, we have HDF5 support to store tensors with a simple schema. HDF5 is universal and has good Python support. We can define a standardized schema to store all different types of data.