openPMD-standard
openPMD-standard copied to clipboard
Proposed: make option "recordBased" for iterationEncoding
It's not comfortable to make separate file/group for each iteration in case of active working with time series (e.g. if required to select single point from every iteration). Much better to make n+1
-dimensional array and write data to it. But, it breaks openPMD compatibility.
Hi @cosmonaut-ok,
Thank you for the idea!
We plan something very similar, currently drafted as "variableBased" encoding related to #221. In that, we will use the intrinsic capabilities of e.g. the ADIOS2 format to encode steps inside a data set (variable) if the file format supports this.
n+1
dimensionality in other formats, e.g. HDF5, is thinkable as a work-around for those formats. Yet we need to be aware that contrary to ADIOS1/2 "step" encoding, HDF5 would then be limited to stay with the same shape of the data set for all iterations; which is limiting for some applications (e.g. particle data shape changes over time).