C3D model crashes with shape mismatch

[igordertigor]

Hi,

I'm trying to run the c3d model from the keras model zoo, but it doesn't even initialize. When I do:

 from kerasmodelzoo.models import c3d
 M = c3d.model(weights=True, summary=True)

The model crashed in line 33 of the c3d module when it tries to add a MaxPooling3D layer. Here is the tail of the backtrace:

Using TensorFlow backend.
Traceback (most recent call last):
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/common_shapes.py", line 594, in call_cpp_shape_fn
    status)
  File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/errors.py", line 463, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors.InvalidArgumentError: Negative dimension size caused by subtracting 2 from 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "load_c3d.py", line 9, in <module>
    model = c3d.model(weights=True, summary=True)
  File "/home/ingo/code/keras-model-zoo/kerasmodelzoo/models/c3d.py", line 34, in model
    border_mode='valid', name='pool3'))
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/models.py", line 324, in add
    output_tensor = layer(self.outputs[0])
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/engine/topology.py", line 517, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/engine/topology.py", line 571, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/engine/topology.py", line 155, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/layers/pooling.py", line 305, in call
    dim_ordering=self.dim_ordering)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/layers/pooling.py", line 353, in _pooling_function
    border_mode, dim_ordering, pool_mode='max')
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 1847, in pool3d
    x = tf.nn.max_pool3d(x, pool_size, strides, padding=padding)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1467, in max_pool3d
    strides=strides, padding=padding, name=name)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 749, in apply_op
    op_def=op_def)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2382, in create_op
    set_shapes_for_outputs(ret)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1783, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/home/ingo/.virtualenvs/k2tf/lib/python3.5/site-packages/tensorflow/python/framework/common_shapes.py", line 596, in call_cpp_shape_fn
    raise ValueError(err.message)
ValueError: Negative dimension size caused by subtracting 2 from 1

The error message is pretty clear but it really happens in the depth of the keras backend and it's not quite obvious which statement actually resulted in this dimension mismatch. I went to the place where the tensorflow max-pooling operation is actually created by the keras backend and looked at the dimensions. Pooling is actually applied with a pool size of (1, 2, 2, 2 , 1) and strides of (1, 2, 2, 2, 1) to a relu-tensor of shape (?, 1, 4, 28, 256). The error probably results from the second dimensions.

I suspected that this might be due to differences in dimension ordering between tensorflow and theano and tried with theano as well. Indeed it gets further but then crashes in line 56 when the Flatten() operation is added. Given that I am not really interested in getting it to run with the theano backend, I didn't investigate any further.

Did anybody have success running the c3d model with tensorflow? Am I missing something important? Any help would be greatly appreciated.

[igordertigor]

Ok, I have this working. I have patched the c3d model to take the correct input dimensions depending on you dim_ordering setting. For loading the model weights, I also had to patch keras to reshuffle the weights. So the solution isn't really clean.

[liamheng]

How do you solve this problem? I have added data_format='channels_first' in each layer, such that the shape mismatch is solved. But when I run c3d_model.load_weights(filepath), it will result in KeyError: "Can't open attribute (Can't locate attribute: 'layer_names')".

[igordertigor]

Hi @lihengbit as I said above, I patched keras to reshuffle the weights. It's quite some time ago and I haven't used it since. So it would be a bit more work to actually dig it out again. Also, I feel that your code has diverged a bit from my solution anyway. Could you at least provide a bit more detail about your error message?

[loopjockey]

@lihengbit I'm hitting the same issue, any luck?

[binhhoangtieu]

I got the same error when I use: model.load_weights('c3d-sports1M_weights.h5')

Traceback (most recent call last): File "3dcnn.py", line 262, in main() File "3dcnn.py", line 137, in main model.load_weights('c3d-sports1M_weights.h5') File "/home/nghialuan/.local/lib/python3.6/site-packages/keras/models.py", line 738, in load_weights topology.load_weights_from_hdf5_group(f, layers, reshape=reshape) File "/home/nghialuan/.local/lib/python3.6/site-packages/keras/engine/topology.py", line 3149, in load_weights_from_hdf5_group layer_names = [n.decode('utf8') for n in f.attrs['layer_names']] File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "/home/nghialuan/.local/lib/python3.6/site-packages/h5py/_hl/attrs.py", line 60, in getitem attr = h5a.open(self._id, self._e(name)) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5a.pyx", line 77, in h5py.h5a.open KeyError: "Can't open attribute (can't locate attribute: 'layer_names')"

Anyone has solution?