Shape inconsistent error

[zhaozhixu]

Hi, I was trying to train your squeezeDetMX on 4 Nvidia Titan Xps, using python 3.6, centOS linux 7, and I encountered a shape inconsistent error. Can you tell me how to fix it? Thanks!

Error details: [Ironboy@localhost squeezeDetMX]$ ~/bin/python3 train.py infer_shape error. Arguments: image: (5, 3, 384, 1248) label_box: (5, 36, 22, 76) label_score: (5, 9, 22, 76) label_class: (5, 27, 22, 76) Traceback (most recent call last): File "train.py", line 59, in main() File "train.py", line 42, in main ctx=[mx.gpu(0), mx.gpu(1), mx.gpu(2), mx.gpu(3)]) File "/data1/Ironboy/source/squeezeDetMX/squeezeDetMX/utils.py", line 53, in build_module inputs_need_grad=inputs_need_grad) File "/data1/Ironboy/lib/python3.6/site-packages/mxnet/module/module.py", line 388, in bind state_names=self._state_names) File "/data1/Ironboy/lib/python3.6/site-packages/mxnet/module/executor_group.py", line 205, in init self.bind_exec(data_shapes, label_shapes, shared_group) File "/data1/Ironboy/lib/python3.6/site-packages/mxnet/module/executor_group.py", line 301, in bind_exec shared_group)) File "/data1/Ironboy/lib/python3.6/site-packages/mxnet/module/executor_group.py", line 548, in _bind_ith_exec arg_shapes, _, aux_shapes = self.symbol.infer_shape(**input_shapes) File "/data1/Ironboy/lib/python3.6/site-packages/mxnet/symbol.py", line 747, in infer_shape res = self._infer_shape_impl(False, *args, **kwargs) File "/data1/Ironboy/lib/python3.6/site-packages/mxnet/symbol.py", line 871, in _infer_shape_impl ctypes.byref(complete))) File "/data1/Ironboy/lib/python3.6/site-packages/mxnet/base.py", line 84, in check_call raise MXNetError(py_str(_LIB.MXGetLastError())) mxnet.base.MXNetError: Error in operator softmaxoutput0: Shape inconsistent, Provided=(3,45,22,76), inferred shape=(3,45,23) Segmentation fault (core dumped)

[MaxBareiss]

This error is also present when adapted to 1 GPU on MXnet 0.12.0.