tftables
tftables copied to clipboard
ZeroDivisionError when `FileReader.get_batch` is called without `block_size`
I get a ZeroDivisionError
when I try to read a batch without specifying what the block_size
is. My HDF5 file is created with h5py and is a simple table.
Here is my code and the error message:
In [2]: reader = tftables.open_file('/home/yngve/table.h5', batch_size=5)
In [3]: a = reader.get_batch('/train/images', n_procs=3)
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
<ipython-input-3-05142b89f50c> in <module>()
----> 1 a = reader.get_batch('/train/images', n_procs=3)
~/anaconda3/envs/tf/lib/python3.6/site-packages/tftables.py in get_batch(self, path, **kw_args)
264 block_size = queue.block_size
265 # get an example for finding data types and row sizes.
--> 266 example = self.streamer.get_remainder(path, block_size)
267 batch_type = example.dtype
268 inner_shape = example.shape[1:]
~/anaconda3/envs/tf/lib/python3.6/site-packages/multitables.py in get_remainder(self, path, block_size)
459 :return: A copy of the remainder elements as a numpy array.
460 """
--> 461 return self.__get_batch(path, length=block_size, last=True)
462
463 class Queue:
~/anaconda3/envs/tf/lib/python3.6/site-packages/multitables.py in __get_batch(self, path, length, last)
444
445 if last:
--> 446 example = h5_node[length*(len(h5_node)//length):].copy()
447 else:
448 example = h5_node[:length].copy()
ZeroDivisionError: integer division or modulo by zero
The error is resolved if I specify block_size
. The code
In [2]: reader = tftables.open_file('/home/yngve/table.h5', batch_size=5)
In [3]: a = reader.get_batch('/train/images', block_size=5, n_procs=3)
does not give any errors. I have tested this code with the FIFOQueue and it does indeed give the expected result.
System info: OS: Ubuntu 16.04 Python version: 3.6 Tensorflow version: 1.5.0 Tftables version: 1.1.2 (latest from pip)
This definitely seems like a bug.
Thanks for the report!