DALI icon indicating copy to clipboard operation
DALI copied to clipboard

if numpy array has been write into tfrecord,how do we can decode it by dali

Open xiaoyang-coder opened this issue 3 years ago • 3 comments

` def _bytes_feature(self, value): return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

feature[key] = self._bytes_feature(np.array(data, dtype=np.float32).tobytes()) example_proto = tf.train.Example(features=tf.train.Features(feature=feature)) tf_writer.write(example_proto.SerializeToString()) ` Thank you very much.

xiaoyang-coder avatar Mar 09 '22 09:03 xiaoyang-coder

Hi @xiaoyang-coder,

DALI doesn't provide a standalone NumPy decoder. If you want to store data inside TFRecord please do it as a raw memory, not a NumPy array.

JanuszL avatar Mar 09 '22 09:03 JanuszL

@JanuszL Thank you for your reply, will the NumPy decoder be provide in the near future? It may be useful feature and very practical。

xiaoyang-coder avatar Mar 09 '22 09:03 xiaoyang-coder

We don't have such a plan, however, we would be more than happy to review and accept any PR introducing this functionality. It can be based on already available numpy reader and loader. In the meantime, you can check on how DALI can be easily extended by user-provided functionalities.

JanuszL avatar Mar 09 '22 09:03 JanuszL