NVIDIA
Assert on "meta" failed: Operator Reader not found or does not expose valid metadata.
Describe the question.
I cannot solve this problem. Could you please tell me how to solve this problem?
class DALIPipeline(Pipeline): def init(self, batch_size, num_threads, device_id, external_data, rsz_w, rsz_h): super(DALIPipeline, self).init(batch_size, num_threads, device_id)
self.external_data = external_data
self.iterator = iter(external_data)
self.input = ops.ExternalSource()
self.input_label = ops.ExternalSource()
self.decode = ops.decoders.Image(device = "mixed", output_type = types.RGB)
#self.decode = ops.ImageDecoderRandomCrop(device = "mixed", output_type = types.DALIImageType.RGB)
self.resize = ops.Resize(device="gpu", resize_x=rsz_w, resize_y=rsz_h, interp_type=types.INTERP_TRIANGULAR)
self.transpose = ops.Transpose(device='gpu', perm = [2, 0, 1])
self.cast = ops.Cast(device='gpu', dtype=types.DALIDataType.FLOAT)
def define_graph(self):
self.jpegs = self.input(name='Reader')
self.label = self.input_label()
img = self.decode(self.jpegs)
img = self.resize(img)
img = self.cast(img)
img = self.transpose(img)
return (img.gpu(), self.label.gpu())
def iter_setup(self):
try:
p = self.iterator.next()
except StopIteration:
self.iterator = iter(self.external_data)
p = self.iterator.next()
img, label = p
self.feed_input(self.jpegs, img)
self.feed_input(self.label, label)
pipe = DALIPipeline(batch_size, num_threads=2, device_id=0, external_data=dataloader, rsz_h=rsz_h, rsz_w=rsz_w)
pipe.build()
dali_iter = DALIGenericIterator(pipelines=pipe,
output_map=['input', 'cls'],
reader_name='Reader',
auto_reset=True,
dynamic_shape=False)
error code:
Traceback (most recent call last):
File "./train_dali.py", line 253, in
Check for duplicates
- [X] I have searched the open bugs/issues and have found no duplicates for this bug report
Hi @ri0107pr,
Thank you for bringing this up. In case of the external source operator there is no notion of number of samples inside it as the callback provided to it is free to run until a StopIteration exception is raised. In that case, please remove reader_name='Reader' from the iterator. The default size set to -1 should guarantee iterator runs until the external source runs out of data.
Hi @ri0107pr
the External Source operator is not a "reader", hence the integration via reader_name parameter with DALIGenericIterator doesn't work.
If you use external source and want to make it work with the DALIGenericIterator I advise to use the source parameter of external source instead of the feed_input method.
https://docs.nvidia.com/deeplearning/dali/user-guide/docs/operations/nvidia.dali.fn.external_source.html
The source can be a callback or iterable, the parameter batch to the external source specifies if you are producing one sample at a time or one batch at a time.
Raising StopIteration from such callback would indicate end of epoch for the DALIGenericIterator.
Of course you would need to remove the reader_name parameter as @JanuszL mentioned.
@JanuszL , @klecki Thank you very much.
I will reread "external_source". There is a possibility that I will ask about the error again. Please let me know if it does.