apollo icon indicating copy to clipboard operation
apollo copied to clipboard
verified publisher icon ApolloAuto

The perception moudle is failed to launch in the Dreamview

Open HungFrancis opened this issue 2 years ago • 2 comments

Hi Community, I face the problem to activate the perception module when following the documents at apollo/docs/06_Perception/how_to_run_perception_module_on_your_local_computer.md. the button of perception is launched, but then close automatically after few seconds, the module seems fail to launch. Your help will be greatly appreciated! Thanks.

System information

  • Linux Ubuntu 20.04:
  • Apollo installed from: source
  • Apollo version: Apollo 8.0
  • Output of apollo.sh config if on master branch:
[INFO] Apollo Environment Settings:
[INFO]     APOLLO_ROOT_DIR:   /apollo
[INFO]     APOLLO_CACHE_DIR:  /apollo/.cache
[INFO]     APOLLO_IN_DOCKER:  true
[INFO]     APOLLO_VERSION:    master-2023-07-14-a3c851fc58
[INFO]     DOCKER_IMG:        dev-x86_64-18.04-20221124_1708
[INFO]     APOLLO_ENV:        STAGE=dev USE_ESD_CAN=false
[INFO]     USE_GPU_HOST:      1
[INFO]     USE_GPU_TARGET:    1
[INFO]     GPU_PLATFORM:      NVIDIA
[INFO] Configure .apollo.bazelrc in non-interactive mode
[INFO] You have bazel 3.7.1 installed.
[INFO] Found possible Python library paths:
       /usr/lib/python3/dist-packages
       /usr/local/lib/python3.6/dist-packages
[INFO] Found CUDA 11.1 in:
       /usr/local/cuda-11.1/targets/x86_64-linux/lib
       /usr/local/cuda-11.1/targets/x86_64-linux/include
[INFO] Found cuDNN 8 in:
       /usr/lib/x86_64-linux-gnu
       /usr/include
[INFO] Found TensorRT 7 in:
       /usr/lib/x86_64-linux-gnu
       /usr/include/x86_64-linux-gnu
[ OK ] Successfully configured .apollo.bazelrc in non-interactive mode.

Steps to reproduce the issue:

Follow the documents at apollo/docs/06_Perception/how_to_run_perception_module_on_your_local_computer.md.

  1. Run bootstrap.sh
bootstrap.sh
  1. Launch static transform
cyber_launch start /apollo/modules/transform/launch/static_transform.launch
  1. Launch the perception modules
cyber_launch start /apollo/modules/perception/production/launch/perception.launch

On the Dreamview, I can see the button of perception is launched, but then close automatically after few seconds.

Supporting materials (screenshots, command lines, code/script snippets):

  • Log when launching the perception modules in the terminal:
[cyber_launch_22311] INFO Launch file [/apollo/modules/perception/production/launch/perception.launch]
[cyber_launch_22311] INFO ========================================================================================================================
[cyber_launch_22311] INFO Load module [perception_lidar] library: [perception_lidar] [CYBER_DEFAULT] conf: [/apollo/modules/perception/production/dag/dag_streaming_perception.dag] exception_handler: []
[cyber_launch_22311] INFO Start process [perception_lidar] successfully. pid: 22318
[cyber_launch_22311] INFO ------------------------------------------------------------------------------------------------------------------------
[cyber_launch_22311] INFO Load module [motion_service] library: [motion_service] [CYBER_DEFAULT] conf: [/apollo/modules/perception/production/dag/dag_motion_service.dag] exception_handler: []
[cyber_launch_22311] INFO Start process [motion_service] successfully. pid: 22320
[cyber_launch_22311] INFO ------------------------------------------------------------------------------------------------------------------------
[perception_lidar]  WARNING: Logging before InitGoogleLogging() is written to STDERR
[motion_service]  WARNING: Logging before InitGoogleLogging() is written to STDERR
[perception_lidar]  I0814 11:41:08.820861 22318 module_argument.cc:81] []command: mainboard -d /apollo/modules/perception/production/dag/dag_streaming_perception.dag -p perception_lidar -s CYBER_DEFAULT 
[motion_service]  I0814 11:41:08.820861 22320 module_argument.cc:81] []command: mainboard -d /apollo/modules/perception/production/dag/dag_motion_service.dag -p motion_service -s CYBER_DEFAULT 
[motion_service]  I0814 11:41:08.822371 22320 global_data.cc:153] []host ip: 192.168.88.241
[perception_lidar]  I0814 11:41:08.822373 22318 global_data.cc:153] []host ip: 192.168.88.241
[perception_lidar]  I0814 11:41:08.822755 22318 module_argument.cc:57] []binary_name_ is mainboard, process_group_ is perception_lidar, has 1 dag conf
[perception_lidar]  I0814 11:41:08.822760 22318 module_argument.cc:60] []dag_conf: /apollo/modules/perception/production/dag/dag_streaming_perception.dag
[motion_service]  I0814 11:41:08.822816 22320 module_argument.cc:57] []binary_name_ is mainboard, process_group_ is motion_service, has 1 dag conf
[motion_service]  I0814 11:41:08.822824 22320 module_argument.cc:60] []dag_conf: /apollo/modules/perception/production/dag/dag_motion_service.dag
[motion_service]  E0814 11:41:08.837893 22320 class_loader_manager.h:70] [mainboard]Invalid class name: MotionService
[motion_service]  E0814 11:41:08.837908 22320 module_controller.cc:67] [mainboard]Failed to load module: /apollo/modules/perception/production/dag/dag_motion_service.dag
[motion_service]  E0814 11:41:08.838969 22320 mainboard.cc:39] [mainboard]module start error.
[motion_service]  
[perception_lidar]  --- Running analysis [ir_graph_build_pass]
[perception_lidar]  --- Running analysis [ir_analysis_pass]
[perception_lidar]  --- Running IR pass [is_test_pass]
[perception_lidar]  --- Running IR pass [simplify_with_basic_ops_pass]
[perception_lidar]  --- Running IR pass [conv_bn_fuse_pass]
[perception_lidar]  WARNING: Logging before InitGoogleLogging() is written to STDERR
[perception_lidar]  I0814 11:41:11.499308 22318 fuse_pass_base.cc:59] ---  detected 99 subgraphs
[perception_lidar]  --- Running IR pass [conv_eltwiseadd_bn_fuse_pass]
[perception_lidar]  --- Running IR pass [embedding_eltwise_layernorm_fuse_pass]
[perception_lidar]  --- Running IR pass [multihead_matmul_fuse_pass_v2]
[perception_lidar]  --- Running IR pass [fused_multi_transformer_encoder_pass]
[perception_lidar]  --- Running IR pass [fused_multi_transformer_decoder_pass]
[perception_lidar]  --- Running IR pass [fused_multi_transformer_encoder_fuse_qkv_pass]
[perception_lidar]  --- Running IR pass [fused_multi_transformer_decoder_fuse_qkv_pass]
[perception_lidar]  --- Running IR pass [multi_devices_fused_multi_transformer_encoder_fuse_qkv_pass]
[perception_lidar]  --- Running IR pass [multi_devices_fused_multi_transformer_decoder_fuse_qkv_pass]
[perception_lidar]  --- Running IR pass [fuse_multi_transformer_layer_pass]
[perception_lidar]  --- Running IR pass [gpu_cpu_squeeze2_matmul_fuse_pass]
[perception_lidar]  --- Running IR pass [gpu_cpu_reshape2_matmul_fuse_pass]
[perception_lidar]  --- Running IR pass [gpu_cpu_flatten2_matmul_fuse_pass]
[perception_lidar]  --- Running IR pass [gpu_cpu_map_matmul_v2_to_mul_pass]
[perception_lidar]  I0814 11:41:13.616678 22318 fuse_pass_base.cc:59] ---  detected 98 subgraphs
[perception_lidar]  --- Running IR pass [gpu_cpu_map_matmul_v2_to_matmul_pass]
[perception_lidar]  I0814 11:41:13.624825 22318 fuse_pass_base.cc:59] ---  detected 25 subgraphs
[perception_lidar]  --- Running IR pass [matmul_scale_fuse_pass]
[perception_lidar]  --- Running IR pass [multihead_matmul_fuse_pass_v3]
[perception_lidar]  --- Running IR pass [gpu_cpu_map_matmul_to_mul_pass]
[perception_lidar]  --- Running IR pass [fc_fuse_pass]
[perception_lidar]  I0814 11:41:13.944676 22318 fuse_pass_base.cc:59] ---  detected 98 subgraphs
[perception_lidar]  --- Running IR pass [fc_elementwise_layernorm_fuse_pass]
[perception_lidar]  I0814 11:41:13.977167 22318 fuse_pass_base.cc:59] ---  detected 6 subgraphs
[perception_lidar]  --- Running IR pass [conv_elementwise_add_fuse_pass]
[perception_lidar]  I0814 11:41:14.097723 22318 fuse_pass_base.cc:59] ---  detected 123 subgraphs
[perception_lidar]  --- Running IR pass [transpose_flatten_concat_fuse_pass]
[perception_lidar]  --- Running IR pass [constant_folding_pass]
[perception_lidar]  --- Running IR pass [auto_mixed_precision_pass]
[perception_lidar]  --- Running IR pass [runtime_context_cache_pass]
[perception_lidar]  --- Running analysis [ir_params_sync_among_devices_pass]
[perception_lidar]  I0814 11:41:14.407845 22318 ir_params_sync_among_devices_pass.cc:89] Sync params from CPU to GPU
[perception_lidar]  --- Running analysis [adjust_cudnn_workspace_size_pass]
[perception_lidar]  --- Running analysis [inference_op_replace_pass]
[perception_lidar]  --- Running analysis [memory_optimize_pass]
[perception_lidar]  I0814 11:41:14.546689 22318 memory_optimize_pass.cc:219] Cluster name : img2lidars  size: 384
[perception_lidar]  I0814 11:41:14.546702 22318 memory_optimize_pass.cc:219] Cluster name : linear_109.tmp_1  size: 36000
[perception_lidar]  I0814 11:41:14.546705 22318 memory_optimize_pass.cc:219] Cluster name : relu_3.tmp_0  size: 49152000
[perception_lidar]  I0814 11:41:14.546705 22318 memory_optimize_pass.cc:219] Cluster name : img  size: 18432000
[perception_lidar]  I0814 11:41:14.546706 22318 memory_optimize_pass.cc:219] Cluster name : relu_2.tmp_0  size: 49152000
[perception_lidar]  I0814 11:41:14.546707 22318 memory_optimize_pass.cc:219] Cluster name : batch_norm_20.tmp_2  size: 49152000
[perception_lidar]  I0814 11:41:14.546707 22318 memory_optimize_pass.cc:219] Cluster name : relu_6.tmp_0  size: 49152000
[perception_lidar]  I0814 11:41:14.546708 22318 memory_optimize_pass.cc:219] Cluster name : concat_0.tmp_0  size: 294912000
[perception_lidar]  I0814 11:41:14.546710 22318 memory_optimize_pass.cc:219] Cluster name : softmax_7.tmp_0  size: 172800000
[perception_lidar]  I0814 11:41:14.546710 22318 memory_optimize_pass.cc:219] Cluster name : relu_4.tmp_0  size: 49152000
[perception_lidar]  I0814 11:41:14.546711 22318 memory_optimize_pass.cc:219] Cluster name : layer_norm_9.tmp_2  size: 921600
[perception_lidar]  I0814 11:41:14.546712 22318 memory_optimize_pass.cc:219] Cluster name : sin_0.tmp_0  size: 1536000
[perception_lidar]  I0814 11:41:14.546712 22318 memory_optimize_pass.cc:219] Cluster name : layer_norm_25.tmp_2  size: 921600
[perception_lidar]  I0814 11:41:14.546713 22318 memory_optimize_pass.cc:219] Cluster name : transpose_79.tmp_0  size: 921600
[perception_lidar]  I0814 11:41:14.546715 22318 memory_optimize_pass.cc:219] Cluster name : layer_norm_21.tmp_2  size: 921600
[perception_lidar]  I0814 11:41:14.546715 22318 memory_optimize_pass.cc:219] Cluster name : tile_3.tmp_0  size: 921600
[perception_lidar]  I0814 11:41:14.546716 22318 memory_optimize_pass.cc:219] Cluster name : layer_norm_17.tmp_2  size: 921600
[perception_lidar]  I0814 11:41:14.546717 22318 memory_optimize_pass.cc:219] Cluster name : layer_norm_13.tmp_2  size: 921600
[perception_lidar]  --- Running analysis [ir_graph_to_program_pass]
[perception_lidar]  I0814 11:41:14.943996 22318 analysis_predictor.cc:1318] ======= optimize end =======
[perception_lidar]  I0814 11:41:14.979591 22318 naive_executor.cc:110] ---  skip [feed], feed -> img2lidars
[perception_lidar]  I0814 11:41:14.979605 22318 naive_executor.cc:110] ---  skip [feed], feed -> img
[perception_lidar]  I0814 11:41:14.989565 22318 naive_executor.cc:110] ---  skip [_generated_var_4], fetch -> fetch
[perception_lidar]  I0814 11:41:14.989578 22318 naive_executor.cc:110] ---  skip [_generated_var_9], fetch -> fetch
[perception_lidar]  I0814 11:41:14.989579 22318 naive_executor.cc:110] ---  skip [_generated_var_14], fetch -> fetch
[perception_lidar]  I0814 11:41:14.989596 22318 pipeline.cc:84] [mainboard]Create stage type : BEV_OBSTACLE_DETECTOR success!
[perception_lidar]  I0814 11:41:14.989606 22318 pipeline.cc:101] [mainboard]OMT_BEV_OBSTACLE_TRACKER already created
[perception_lidar]  I0814 11:41:14.989607 22318 pipeline.cc:101] [mainboard]OMT_BEV_OBSTACLE_TRACKER already created
[perception_lidar]  I0814 11:41:14.989609 22318 pipeline.cc:101] [mainboard]OMT_BEV_OBSTACLE_TRACKER already created
[perception_lidar]  W0814 11:41:14.989637 22318 environment.h:33] [mainboard]Environment variable [MODULE_PATH] not set, fallback to 
[perception_lidar]
  • Using the mainboard to check the error relate to motion_service:
$ mainboard -d /apollo/modules/perception/production/dag/dag_motion_service.dag -p motion_service -s CYBER_DEFAULT].

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0814 11:45:42.437476 22601 module_argument.cc:81] []command: mainboard -d /apollo/modules/perception/production/dag/dag_motion_service.dag -p motion_service -s CYBER_DEFAULT]. 
I0814 11:45:42.438320 22601 global_data.cc:153] []host ip: 192.168.88.241
I0814 11:45:42.438686 22601 module_argument.cc:57] []binary_name_ is mainboard, process_group_ is motion_service, has 1 dag conf
I0814 11:45:42.438691 22601 module_argument.cc:60] []dag_conf: /apollo/modules/perception/production/dag/dag_motion_service.dag
E0814 11:45:42.457309 22601 class_loader_manager.h:70] [mainboard]Invalid class name: MotionService
E0814 11:45:42.457326 22601 module_controller.cc:67] [mainboard]Failed to load module: /apollo/modules/perception/production/dag/dag_motion_service.dag
E0814 11:45:42.458173 22601 mainboard.cc:39] [mainboard]module start error.
  • Using the mainboard to check the error relate to perception:
$ mainboard -d /apollo/modules/perception/production/dag/dag_streaming_perception.dag -p perception_lidar -s CYBER_DEFAULT].
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0814 11:43:30.772852 22488 module_argument.cc:81] []command: mainboard -d /apollo/modules/perception/production/dag/dag_streaming_perception.dag -p perception_lidar -s CYBER_DEFAULT]. 
I0814 11:43:30.773690 22488 global_data.cc:153] []host ip: 192.168.88.241
I0814 11:43:30.774075 22488 module_argument.cc:57] []binary_name_ is mainboard, process_group_ is perception_lidar, has 1 dag conf
I0814 11:43:30.774080 22488 module_argument.cc:60] []dag_conf: /apollo/modules/perception/production/dag/dag_streaming_perception.dag
--- Running analysis [ir_graph_build_pass]
--- Running analysis [ir_analysis_pass]
--- Running IR pass [is_test_pass]
--- Running IR pass [simplify_with_basic_ops_pass]
--- Running IR pass [conv_bn_fuse_pass]
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0814 11:43:33.369827 22488 fuse_pass_base.cc:59] ---  detected 99 subgraphs
--- Running IR pass [conv_eltwiseadd_bn_fuse_pass]
--- Running IR pass [embedding_eltwise_layernorm_fuse_pass]
--- Running IR pass [multihead_matmul_fuse_pass_v2]
--- Running IR pass [fused_multi_transformer_encoder_pass]
--- Running IR pass [fused_multi_transformer_decoder_pass]
--- Running IR pass [fused_multi_transformer_encoder_fuse_qkv_pass]
--- Running IR pass [fused_multi_transformer_decoder_fuse_qkv_pass]
--- Running IR pass [multi_devices_fused_multi_transformer_encoder_fuse_qkv_pass]
--- Running IR pass [multi_devices_fused_multi_transformer_decoder_fuse_qkv_pass]
--- Running IR pass [fuse_multi_transformer_layer_pass]
--- Running IR pass [gpu_cpu_squeeze2_matmul_fuse_pass]
--- Running IR pass [gpu_cpu_reshape2_matmul_fuse_pass]
--- Running IR pass [gpu_cpu_flatten2_matmul_fuse_pass]
--- Running IR pass [gpu_cpu_map_matmul_v2_to_mul_pass]
I0814 11:43:35.371776 22488 fuse_pass_base.cc:59] ---  detected 98 subgraphs
--- Running IR pass [gpu_cpu_map_matmul_v2_to_matmul_pass]
I0814 11:43:35.380401 22488 fuse_pass_base.cc:59] ---  detected 25 subgraphs
--- Running IR pass [matmul_scale_fuse_pass]
--- Running IR pass [multihead_matmul_fuse_pass_v3]
--- Running IR pass [gpu_cpu_map_matmul_to_mul_pass]
--- Running IR pass [fc_fuse_pass]
I0814 11:43:35.683177 22488 fuse_pass_base.cc:59] ---  detected 98 subgraphs
--- Running IR pass [fc_elementwise_layernorm_fuse_pass]
I0814 11:43:35.714390 22488 fuse_pass_base.cc:59] ---  detected 6 subgraphs
--- Running IR pass [conv_elementwise_add_fuse_pass]
I0814 11:43:35.824373 22488 fuse_pass_base.cc:59] ---  detected 123 subgraphs
--- Running IR pass [transpose_flatten_concat_fuse_pass]
--- Running IR pass [constant_folding_pass]
--- Running IR pass [auto_mixed_precision_pass]
--- Running IR pass [runtime_context_cache_pass]
--- Running analysis [ir_params_sync_among_devices_pass]
I0814 11:43:36.134789 22488 ir_params_sync_among_devices_pass.cc:89] Sync params from CPU to GPU
--- Running analysis [adjust_cudnn_workspace_size_pass]
--- Running analysis [inference_op_replace_pass]
--- Running analysis [memory_optimize_pass]
I0814 11:43:36.278972 22488 memory_optimize_pass.cc:219] Cluster name : img2lidars  size: 384
I0814 11:43:36.278993 22488 memory_optimize_pass.cc:219] Cluster name : linear_109.tmp_1  size: 36000
I0814 11:43:36.278995 22488 memory_optimize_pass.cc:219] Cluster name : relu_3.tmp_0  size: 49152000
I0814 11:43:36.278998 22488 memory_optimize_pass.cc:219] Cluster name : img  size: 18432000
I0814 11:43:36.279001 22488 memory_optimize_pass.cc:219] Cluster name : relu_2.tmp_0  size: 49152000
I0814 11:43:36.279003 22488 memory_optimize_pass.cc:219] Cluster name : batch_norm_20.tmp_2  size: 49152000
I0814 11:43:36.279011 22488 memory_optimize_pass.cc:219] Cluster name : relu_6.tmp_0  size: 49152000
I0814 11:43:36.279012 22488 memory_optimize_pass.cc:219] Cluster name : concat_0.tmp_0  size: 294912000
I0814 11:43:36.279016 22488 memory_optimize_pass.cc:219] Cluster name : softmax_7.tmp_0  size: 172800000
I0814 11:43:36.279018 22488 memory_optimize_pass.cc:219] Cluster name : relu_4.tmp_0  size: 49152000
I0814 11:43:36.279021 22488 memory_optimize_pass.cc:219] Cluster name : layer_norm_9.tmp_2  size: 921600
I0814 11:43:36.279024 22488 memory_optimize_pass.cc:219] Cluster name : sin_0.tmp_0  size: 1536000
I0814 11:43:36.279029 22488 memory_optimize_pass.cc:219] Cluster name : layer_norm_25.tmp_2  size: 921600
I0814 11:43:36.279032 22488 memory_optimize_pass.cc:219] Cluster name : transpose_79.tmp_0  size: 921600
I0814 11:43:36.279034 22488 memory_optimize_pass.cc:219] Cluster name : layer_norm_21.tmp_2  size: 921600
I0814 11:43:36.279038 22488 memory_optimize_pass.cc:219] Cluster name : tile_3.tmp_0  size: 921600
I0814 11:43:36.279040 22488 memory_optimize_pass.cc:219] Cluster name : layer_norm_17.tmp_2  size: 921600
I0814 11:43:36.279043 22488 memory_optimize_pass.cc:219] Cluster name : layer_norm_13.tmp_2  size: 921600
--- Running analysis [ir_graph_to_program_pass]
I0814 11:43:36.742965 22488 analysis_predictor.cc:1318] ======= optimize end =======
I0814 11:43:36.782217 22488 naive_executor.cc:110] ---  skip [feed], feed -> img2lidars
I0814 11:43:36.782234 22488 naive_executor.cc:110] ---  skip [feed], feed -> img
I0814 11:43:36.792750 22488 naive_executor.cc:110] ---  skip [_generated_var_4], fetch -> fetch
I0814 11:43:36.792763 22488 naive_executor.cc:110] ---  skip [_generated_var_9], fetch -> fetch
I0814 11:43:36.792766 22488 naive_executor.cc:110] ---  skip [_generated_var_14], fetch -> fetch
I0814 11:43:36.792784 22488 pipeline.cc:84] [mainboard]Create stage type : BEV_OBSTACLE_DETECTOR success!
I0814 11:43:36.792798 22488 pipeline.cc:101] [mainboard]OMT_BEV_OBSTACLE_TRACKER already created
I0814 11:43:36.792800 22488 pipeline.cc:101] [mainboard]OMT_BEV_OBSTACLE_TRACKER already created
I0814 11:43:36.792804 22488 pipeline.cc:101] [mainboard]OMT_BEV_OBSTACLE_TRACKER already created
W0814 11:43:36.792843 22488 environment.h:33] [mainboard]Environment variable [MODULE_PATH] not set, fallback to 
Segmentation fault (core dumped)

HungFrancis avatar Aug 14 '23 09:08 HungFrancis

好兄弟你解决了吗，我也有同样的问题。用的是apollo7.0版本 913D655F8CEDDD08C05949A87F282A77 如图代码，我添加了一些随机的车辆和随机的路人，并且在dreamview打开了perception模块，但是为什么在开始模拟之后dreamview上面无法显示这些路人和车辆 11b1bdae62960220fbde20fbf06028e3

lsm2842035890 avatar Nov 13 '23 10:11 lsm2842035890

Not yet. Sorry I am not using the Simulator like you. We should wait for them to answer our issues.

HungFrancis avatar Dec 10 '23 18:12 HungFrancis