asdf-standard
asdf-standard copied to clipboard
Streaming multiple arrays to file
I would like to know how to stream multiple arrays to the same ASDF file. I modified the example to include two different streams in the tree dictionary. How do I write both to the same file?
from asdf import AsdfFile, Stream
import numpy as np
tree = {
# Each "row" of data will have 128 entries.
'my_stream_0': Stream([128], np.float64)
'my_stream_1': Stream([128], np.float64)
}
ff = AsdfFile(tree)
with open('test.asdf', 'wb') as fd:
ff.write_to(fd)
# Write 100 rows of data, one row at a time. ``write``
# expects the raw binary bytes, not an array, so we use
# ``tobytes()``.
for i in range(100):
fd.write(np.array([i] * 128, np.float64).tobytes())
fd.write(np.array([i] * 128, np.float64).tobytes())
I'm curious about the motivations for this, but before delving into this, are you aware of other formats that support this kind of feature? That might help us understand the issues better.