Apollo 8.0 Camera Perception Module fails to start with default settings

[howicc]

Describe the bug

When I launch the Camera Perception Module I get this error related to PyTorch. Other modules are running with no issues. No changes in default settings.

To Reproduce Steps to reproduce the behavior:

1. Launch Perception Camera

cyber_launch start modules/perception/production/launch/perception_camera.launch

or

mainboard -d /apollo/modules/perception/production/dag/dag_streaming_perception.dag

2. See error

Expected behavior Perception Module to be launched and running.

Desktop (please complete the following information):

• OS: Ubuntu 20.04
• Apollo Docker
• Apollo 8.0 (both v8.0.0 and current Master Branch) show the same error.

Additional context

Full Error Log:

[mainboard_default_2326454]  I1028 01:27:52.999181 2326461 module_argument.cc:81] []command: mainboard -d /apollo/modules/perception/production/dag/dag_streaming_perception_camera.dag -d /apollo/modules/perception/production/dag/dag_motion_service.dag -p mainboard_default_2326454 -s CYBER_DEFAULT 
[mainboard_default_2326454]  I1028 01:27:52.999351 2326461 global_data.cc:153] []host ip: 10.213.22.232
[mainboard_default_2326454]  I1028 01:27:53.000300 2326461 module_argument.cc:57] []binary_name_ is mainboard, process_group_ is mainboard_default_2326454, has 2 dag conf
[mainboard_default_2326454]  I1028 01:27:53.000305 2326461 module_argument.cc:60] []dag_conf: /apollo/modules/perception/production/dag/dag_streaming_perception_camera.dag
[mainboard_default_2326454]  I1028 01:27:53.000306 2326461 module_argument.cc:60] []dag_conf: /apollo/modules/perception/production/dag/dag_motion_service.dag
[mainboard_default_2326454]  terminate called after throwing an instance of 'std::runtime_error'
[mainboard_default_2326454]    what():  The following operation failed in the TorchScript interpreter.
[mainboard_default_2326454]  Traceback of TorchScript, serialized code (most recent call last):
[mainboard_default_2326454]    File "code/__torch__/smoke/modeling/detector/keypoint_detector.py", line 14, in forward
[mainboard_default_2326454]      Ks, down_ratios, = argument_2
[mainboard_default_2326454]      _2 = (_1).forward(input, )
[mainboard_default_2326454]      _3 = ((_0).forward(_2, Ks, down_ratios, ), _2)
[mainboard_default_2326454]             ~~~~~~~~~~~ <--- HERE
[mainboard_default_2326454]      return _3
[mainboard_default_2326454]    File "code/__torch__/smoke/modeling/heads/smoke_head/smoke_head.py", line 12, in forward
[mainboard_default_2326454]      down_ratios: Tensor) -> Tensor:
[mainboard_default_2326454]      _0 = self.post_processor
[mainboard_default_2326454]      _1 = (self.predictor).forward(argument_1, )
[mainboard_default_2326454]            ~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
[mainboard_default_2326454]      _2, _3, _4, _5, _6, _7, = _1
[mainboard_default_2326454]      _8 = (_0).forward(_2, _3, _4, _5, _6, _7, Ks, down_ratios, )
[mainboard_default_2326454]    File "code/__torch__/smoke/modeling/heads/smoke_head/smoke_predictor.py", line 19, in forward
[mainboard_default_2326454]      _5 = torch.slice(_2, 0, 0, 9223372036854775807, 1)
[mainboard_default_2326454]      _6 = torch.slice(_5, 1, 3, 6, 1)
[mainboard_default_2326454]      _7 = torch.expand(torch.view(_4, [3, 160, 240]), [1, 3, 160, 240], implicit=True)
[mainboard_default_2326454]                        ~~~~~~~~~~ <--- HERE
[mainboard_default_2326454]      _8 = torch.copy_(_6, _7, False)
[mainboard_default_2326454]      _9 = torch.slice(_2, 0, 0, 9223372036854775807, 1)
[mainboard_default_2326454]  
[mainboard_default_2326454]  Traceback of TorchScript, original code (most recent call last):
[mainboard_default_2326454]  /home/zhangxiao/code/baidu/apollo-bj/apollo-perception/apollo_perception/camera/obstacle/SMOKE/smoke/modeling/heads/smoke_head/smoke_predictor.py(125): forward
[mainboard_default_2326454]  /home/zhangxiao/anaconda3/envs/pytorch1.7/lib/python3.8/site-packages/torch/nn/modules/module.py(709): _slow_forward
[mainboard_default_2326454]  /home/zhangxiao/anaconda3/envs/pytorch1.7/lib/python3.8/site-packages/torch/nn/modules/module.py(725): _call_impl
[mainboard_default_2326454]  /home/zhangxiao/code/baidu/apollo-bj/apollo-perception/apollo_perception/camera/obstacle/SMOKE/smoke/modeling/heads/smoke_head/smoke_head.py(42): forward
[mainboard_default_2326454]  /home/zhangxiao/anaconda3/envs/pytorch1.7/lib/python3.8/site-packages/torch/nn/modules/module.py(709): _slow_forward
[mainboard_default_2326454]  /home/zhangxiao/anaconda3/envs/pytorch1.7/lib/python3.8/site-packages/torch/nn/modules/module.py(725): _call_impl
[mainboard_default_2326454]  /home/zhangxiao/code/baidu/apollo-bj/apollo-perception/apollo_perception/camera/obstacle/SMOKE/smoke/modeling/detector/keypoint_detector.py(49): forward
[mainboard_default_2326454]  /home/zhangxiao/anaconda3/envs/pytorch1.7/lib/python3.8/site-packages/torch/nn/modules/module.py(709): _slow_forward
[mainboard_default_2326454]  /home/zhangxiao/anaconda3/envs/pytorch1.7/lib/python3.8/site-packages/torch/nn/modules/module.py(725): _call_impl
[mainboard_default_2326454]  /home/zhangxiao/anaconda3/envs/pytorch1.7/lib/python3.8/site-packages/torch/jit/_trace.py(934): trace_module
[mainboard_default_2326454]  /home/zhangxiao/anaconda3/envs/pytorch1.7/lib/python3.8/site-packages/torch/jit/_trace.py(733): trace
[mainboard_default_2326454]  tools/torch2onnx.py(59): main
[mainboard_default_2326454]  tools/torch2onnx.py(66): <module>
[mainboard_default_2326454]  RuntimeError: shape '[3, 160, 240]' is invalid for input of size 155520
[mainboard_default_2326454]  

[LordonCN]

Pls to try this model

[howicc]

Thank you. How do I replace Apollo's model with this?

[LordonCN]

Thank you. How do I replace Apollo's model with this?

Under path modules/perception/production/data/perception/camera/models

[howicc]

Thank you. Adding the model solved the issue.