apollo icon indicating copy to clipboard operation
apollo copied to clipboard
verified publisher icon ApolloAuto

Perception Traffic Light module run failed

Open cirobrignolo opened this issue 1 year ago • 3 comments

System information

  • OS Platform and Distribution: Ubuntu 20.04
  • Apollo installed from: source
  • Apollo version: 9.0 (I think it's version 9.0. As of today (Nov 23, 2024), I am using the master branch.)
  • Output of apollo.sh config if on master branch:
[INFO] Apollo Environment Settings:
[INFO]     APOLLO_ROOT_DIR:   /apollo
[INFO]     APOLLO_CACHE_DIR:  /apollo/.cache
[INFO]     APOLLO_IN_DOCKER:  true
[INFO]     APOLLO_VERSION:    master-2024-08-28-fb23f6a152
[INFO]     DOCKER_IMG:        dev-x86_64-18.04-20240620_1444
[INFO]     APOLLO_ENV:        STAGE=dev USE_ESD_CAN=false
[INFO]     USE_GPU_HOST:      1
[INFO]     USE_GPU_TARGET:    1
[INFO]     GPU_PLATFORM:      NVIDIA
[INFO] Configure .apollo.bazelrc in non-interactive mode
[INFO] You have bazel 5.2.0 installed.
[INFO] Found possible Python library paths:
       /usr/local/lib/python3.6/dist-packages
       /usr/lib/python3/dist-packages
[INFO] Found CUDA 11.1 in:
       /usr/local/cuda-11.1/targets/x86_64-linux/lib
       /usr/local/cuda-11.1/targets/x86_64-linux/include
[INFO] Found cuDNN 8 in:
       /usr/lib/x86_64-linux-gnu
       /usr/include
[INFO] Found TensorRT 7 in:
       /usr/lib/x86_64-linux-gnu
       /usr/include/x86_64-linux-gnu
[ OK ] Successfully configured .apollo.bazelrc in non-interactive mode.
[INFO] Usage: /apollo/scripts/apollo_config.sh [Options]
[INFO] Options:
[INFO]     -i|--interactive      Run in interactive mode
[INFO]     -n|--noninteractive   Run in non-interactive mode
[INFO]     -h|--help             Show this message and exit

Problem and steps

First, I will explain the purpose of what I am doing. In the second step, I will outline the steps I followed to install the Apollo project, which might have led to an incorrect installation, and consequently, the error could be related to this. Finally, in the third step, I will show the error, the steps I followed, and the issue itself.

  1. Purpose

What I am trying to do is run only the traffic camera perception process using a custom video. I want to do this to analyze how the traffic camera flow works exclusively and perform tests focused on that. Therefore, my idea is to use a recorded video, simulating a vehicle-mounted camera, and have that video serve as the input for the traffic camera perception process to obtain the corresponding bounding boxes. To achieve this, my plan is to run the perception module, particularly the traffic camera component, and inject the video into that process somehow.

  1. Preinstallation, Installation and run

Initially, I followed the prerequisites, which involved installing the GPU drivers, CUDA, LibTorch, Docker, Docker Engine, and NVIDIA Container Toolkit. After that, I cloned the project and started using it from the master branch. I followed the steps that required running the following commands:

bash docker/scripts/dev_start.sh
bash docker/scripts/dev_into.sh 
./apollo.sh build

Finally, to run it:

./scripts/bootstrap.sh

Then, I installed the demo and ran it with:

cyber_recorder play -f docs/02_Quick\ Start/demo_guide/demo_3.5.record -l

Up to that point, I was able to open Dreamview and see the demo. Since my purpose is related to the perception modules, I installed additional configurations that were prompted during the build, as they needed to be installed separately. These are all the steps I followed prior to running the perception module.

  1. Problem

As I mentioned earlier, I want to run the perception module for traffic cameras. Therefore, I followed the guide:

https://github.com/ApolloAuto/apollo/blob/master/docs/06_Perception/how_to_run_perception_module_on_your_local_computer.md

Following the steps outlined there, the only difference in my process was that instead of running ./apollo.sh build_opt_gpu, I ran ./apollo.sh build. Then, in Dreamview, I selected the following options: setup-mode: Mkz Lgsvl car: Mkz Lgsvl 321 map: Sunnyvale Next, I attempted to run the following command:

cyber_launch start /apollo/modules/transform/launch/static_transform.launch

However, the process never completed, and the output seemed to stall. The output was:

[cyber_launch_8895] INFO Launch file [/apollo/modules/transform/launch/static_transform.launch]
[cyber_launch_8895] INFO ========================================================================================================================
[cyber_launch_8895] INFO Load module [static_transform] library: [mainboard_default_8895] [CYBER_DEFAULT] conf, exception_handler: [], respawn_limit: [3]
[cyber_launch_8895] INFO Start process [mainboard_default_8895] successfully. pid: 8902
[cyber_launch_8895] INFO ------------------------------------------------------------------------------------------------------------------------
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1123 18:29:09.008901  8902 module_argument.cc:113] []command: mainboard -d /apollo/modules/transform/dag/static_transform.dag -p mainboard_default_8895 -s CYBER_DEFAULT 
I1123 18:29:09.009255  8902 global_data.cc:153] []host ip: 192.168.1.6
I1123 18:29:09.011008  8902 module_argument.cc:82] []binary_name_ is mainboard, process_group_ is mainboard_default_8895, has 1 dag conf
I1123 18:29:09.011015  8902 module_argument.cc:85] []dag_conf: /apollo/modules/transform/dag/static_transform.dag
I1123 18:29:09.012453  8921 third_party/var/bvar/variable.cpp:792] Write all bvar to /apollo/dumps/static_transform.dag every 10 seconds.

Since this didn't work, I directly tried launching the perception module with the command:

cyber_launch start /apollo/modules/perception/launch/perception_trafficlight.launch

Note that in my Apollo version, the /production/ folder mentioned in the guide doesn't exist. The output I got was:

[cyber_launch_9081] INFO Launch file [/apollo/modules/perception/launch/perception_trafficlight.launch]
[cyber_launch_9081] INFO ========================================================================================================================
[cyber_launch_9081] INFO Load module [trafficlight] library: [trafficlight] [CYBER_DEFAULT] conf, exception_handler: [], respawn_limit: [3]
[cyber_launch_9081] INFO Start process [trafficlight] successfully. pid: 9088
[cyber_launch_9081] INFO ------------------------------------------------------------------------------------------------------------------------
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1123 18:31:45.456794  9088 module_argument.cc:113] []command: mainboard -d /apollo/modules/perception/traffic_light_region_proposal/dag/traffic_light_region_proposal.dag -d /apollo/modules/perception/traffic_light_detection/dag/traffic_light_detection.dag -d /apollo/modules/perception/traffic_light_recognition/dag/traffic_light_recognition.dag -d /apollo/modules/perception/traffic_light_tracking/dag/traffic_light_tracking.dag -p trafficlight -s CYBER_DEFAULT 
I1123 18:31:45.457144  9088 global_data.cc:153] []host ip: 192.168.1.6
I1123 18:31:45.459453  9088 module_argument.cc:82] []binary_name_ is mainboard, process_group_ is trafficlight, has 4 dag conf
I1123 18:31:45.459463  9088 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_region_proposal/dag/traffic_light_region_proposal.dag
I1123 18:31:45.459467  9088 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_detection/dag/traffic_light_detection.dag
I1123 18:31:45.459471  9088 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_recognition/dag/traffic_light_recognition.dag
I1123 18:31:45.459475  9088 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_tracking/dag/traffic_light_tracking.dag
I1123 18:31:45.460440  9107 third_party/var/bvar/variable.cpp:792] Write all bvar to /apollo/dumps/traffic_light_region_proposal.dag_traffic_light_detection.dag_traffic_light_recognition.dag_traffic_light_tracking.dag every 10 seconds.
E1123 18:31:48.556118  9088 hdmap_common.cc:149] [map]lane[id = id: "1391_1_-1"
]. sampled_left_width_[1.04808] is too small. It should be larger than half vehicle width[1.05].
E1123 18:31:48.556322  9088 hdmap_common.cc:158] [map]lane[id = id: "1391_1_-1"
]. sampled_right_width_[1.04402] is too small. It should be larger than half vehicle width[1.05].
E1123 18:31:48.565845  9088 hdmap_common.cc:158] [map]lane[id = id: "6997a_1_-1"
]. sampled_right_width_[1.03532] is too small. It should be larger than half vehicle width[1.05].
E1123 18:31:48.565871  9088 hdmap_common.cc:158] [map]lane[id = id: "6997a_1_-1"
]. sampled_right_width_[1.03707] is too small. It should be larger than half vehicle width[1.05].
E1123 18:31:48.566929  9088 hdmap_common.cc:149] [map]lane[id = id: "19387a_1_-1"
]. sampled_left_width_[1.00808] is too small. It should be larger than half vehicle width[1.05].
E1123 18:31:48.566954  9088 hdmap_common.cc:149] [map]lane[id = id: "19387a_1_-1"
]. sampled_left_width_[1.00224] is too small. It should be larger than half vehicle width[1.05].
E1123 18:31:48.566969  9088 hdmap_common.cc:149] [map]lane[id = id: "19387a_1_-1"
]. sampled_left_width_[0.989222] is too small. It should be larger than half vehicle width[1.05].
E1123 18:31:48.566982  9088 hdmap_common.cc:149] [map]lane[id = id: "19387a_1_-1"
]. sampled_left_width_[0.984685] is too small. It should be larger than half vehicle width[1.05].
E1123 18:31:48.567013  9088 hdmap_common.cc:149] [map]lane[id = id: "19387a_1_-1"
]. sampled_left_width_[0.985358] is too small. It should be larger than half vehicle width[1.05].

The output contains many errors related to lane width. Skipping repetitive lines for brevity, the process eventually failed with:

E1123 18:31:49.268774  9088 hdmap_common.cc:158] [map]lane[id = id: "7657a_1_-1"
]. sampled_right_width_[0.997895] is too small. It should be larger than half vehicle width[1.05].
E1123 18:31:49.268792  9088 hdmap_common.cc:158] [map]lane[id = id: "7657a_1_-1"
]. sampled_right_width_[1.03294] is too small. It should be larger than half vehicle width[1.05].
E1123 18:31:54.277089  9088 rt_utils.cc:48] [mainboard]cannot open file /apollo/modules/perception/data/models/tl_detection_caffe/baidu_iter_140000.caffemodel
F1123 18:31:54.277103  9088 rt_net.cc:706] [mainboard]open file /apollo/modules/perception/data/models/tl_detection_caffe/baidu_iter_140000.caffemodel failed
*** Check failure stack trace: ***
    @     0x7f80e96d967d  google::LogMessage::Fail()
    @     0x7f80e96dbb3a  google::LogMessage::SendToLog()
    @     0x7f80e96d908d  google::LogMessage::Flush()
    @     0x7f80e96d92ce  google::LogMessage::~LogMessage()
    @     0x7f807a3ecfb1  apollo::perception::inference::RTNet::loadWeights()
    @     0x7f807a3ee731  apollo::perception::inference::RTNet::RTNet()
    @     0x7f807ae3acef  apollo::perception::inference::CreateInferenceByName()
    @     0x7f807b4ea846  apollo::perception::trafficlight::TrafficLightDetection::Init()
    @     0x7f807bc2d3f6  apollo::perception::trafficlight::TrafficLightDetectComponent::InitAlgorithmPlugin()
    @     0x7f807bc2cb2b  apollo::perception::trafficlight::TrafficLightDetectComponent::Init()
    @     0x7f807bc6c1d4  apollo::cyber::Component<>::Initialize()
    @     0x55b67386fea6  apollo::cyber::mainboard::ModuleController::LoadModule()
    @     0x55b673870314  apollo::cyber::mainboard::ModuleController::LoadModule()
    @     0x55b67386f819  apollo::cyber::mainboard::ModuleController::LoadAll()
    @     0x55b67386bd7c  apollo::cyber::mainboard::ModuleController::Init()
    @     0x55b67386a598  main
    @     0x7f80e8585c87  __libc_start_main
    @     0x55b67386a0aa  _start
    @              (nil)  (unknown)
[cyber_launch_9081] ERROR Process [trafficlight] has died [pid 9088, exit code -6, cmd mainboard -d /apollo/modules/perception/traffic_light_region_proposal/dag/traffic_light_region_proposal.dag -d /apollo/modules/perception/traffic_light_detection/dag/traffic_light_detection.dag -d /apollo/modules/perception/traffic_light_recognition/dag/traffic_light_recognition.dag -d /apollo/modules/perception/traffic_light_tracking/dag/traffic_light_tracking.dag -p trafficlight -s CYBER_DEFAULT].
[cyber_launch_9081] INFO All processes has died.
[cyber_launch_9081] INFO Cyber exit.
[cyber_launch_9081] INFO All processes have been stopped.

I also tried with the next configuration: setup-mode: Mkz Lgsvl car: Kitti 140 map: Sunnyvale

Then run the next command:

cyber_launch start /apollo/modules/perception/launch/perception_trafficlight.launch

And the output was:

[cyber_launch_14934] INFO Launch file [/apollo/modules/perception/launch/perception_trafficlight.launch]
[cyber_launch_14934] INFO ========================================================================================================================
[cyber_launch_14934] INFO Load module [trafficlight] library: [trafficlight] [CYBER_DEFAULT] conf, exception_handler: [], respawn_limit: [3]
[cyber_launch_14934] INFO Start process [trafficlight] successfully. pid: 14942
[cyber_launch_14934] INFO ------------------------------------------------------------------------------------------------------------------------
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1123 20:21:22.381016 14942 module_argument.cc:113] []command: mainboard -d /apollo/modules/perception/traffic_light_region_proposal/dag/traffic_light_region_proposal.dag -d /apollo/modules/perception/traffic_light_detection/dag/traffic_light_detection.dag -d /apollo/modules/perception/traffic_light_recognition/dag/traffic_light_recognition.dag -d /apollo/modules/perception/traffic_light_tracking/dag/traffic_light_tracking.dag -p trafficlight -s CYBER_DEFAULT 
I1123 20:21:22.381285 14942 global_data.cc:153] []host ip: 192.168.1.6
I1123 20:21:22.383363 14942 module_argument.cc:82] []binary_name_ is mainboard, process_group_ is trafficlight, has 4 dag conf
I1123 20:21:22.383376 14942 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_region_proposal/dag/traffic_light_region_proposal.dag
I1123 20:21:22.383379 14942 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_detection/dag/traffic_light_detection.dag
I1123 20:21:22.383381 14942 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_recognition/dag/traffic_light_recognition.dag
I1123 20:21:22.383384 14942 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_tracking/dag/traffic_light_tracking.dag
I1123 20:21:22.384325 14963 third_party/var/bvar/variable.cpp:792] Write all bvar to /apollo/dumps/traffic_light_region_proposal.dag_traffic_light_detection.dag_traffic_light_recognition.dag_traffic_light_tracking.dag every 10 seconds.
E1123 20:21:24.952502 14942 multi_camera_projection.cc:51] [trafficlight]sensor front_6mm do not exist
E1123 20:21:24.952518 14942 tl_preprocessor.cc:30] [trafficlight]init multi_camera_projection failed.
E1123 20:21:24.952523 14942 traffic_light_region_proposal_component.cc:132] [trafficlight]TrafficLightsPerceptionComponent init preprocessor failed
E1123 20:21:24.952530 14942 traffic_light_region_proposal_component.cc:54] [trafficlight]TrafficLightsPerceptionComponent InitAlgorithmPlugin failed.
E1123 20:21:24.952534 14942 component.h:157] [mainboard]Component Init() failed.
E1123 20:21:24.952806 14942 module_controller.cc:69] [mainboard]Failed to load module: /apollo/modules/perception/traffic_light_region_proposal/dag/traffic_light_region_proposal.dag
E1123 20:21:24.952859 14942 mainboard.cc:66] [mainboard]module start error.
[cyber_launch_14934] ERROR Process [trafficlight] has died [pid 14942, exit code 255, cmd mainboard -d /apollo/modules/perception/traffic_light_region_proposal/dag/traffic_light_region_proposal.dag -d /apollo/modules/perception/traffic_light_detection/dag/traffic_light_detection.dag -d /apollo/modules/perception/traffic_light_recognition/dag/traffic_light_recognition.dag -d /apollo/modules/perception/traffic_light_tracking/dag/traffic_light_tracking.dag -p trafficlight -s CYBER_DEFAULT].
[cyber_launch_14934] INFO All processes has died.
[cyber_launch_14934] INFO Cyber exit.
[cyber_launch_14934] INFO All processes have been stopped.

And the last configuration I tried: setup-mode: Mkz Lgsvl car: Mkz Example map: Sunnyvale

Then run the next command:

cyber_launch start /apollo/modules/perception/launch/perception_trafficlight.launch

And the output was:

[cyber_launch_14677] INFO Launch file [/apollo/modules/perception/launch/perception_trafficlight.launch]
[cyber_launch_14677] INFO ========================================================================================================================
[cyber_launch_14677] INFO Load module [trafficlight] library: [trafficlight] [CYBER_DEFAULT] conf, exception_handler: [], respawn_limit: [3]
[cyber_launch_14677] INFO Start process [trafficlight] successfully. pid: 14684
[cyber_launch_14677] INFO ------------------------------------------------------------------------------------------------------------------------
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1123 20:20:31.816656 14684 module_argument.cc:113] []command: mainboard -d /apollo/modules/perception/traffic_light_region_proposal/dag/traffic_light_region_proposal.dag -d /apollo/modules/perception/traffic_light_detection/dag/traffic_light_detection.dag -d /apollo/modules/perception/traffic_light_recognition/dag/traffic_light_recognition.dag -d /apollo/modules/perception/traffic_light_tracking/dag/traffic_light_tracking.dag -p trafficlight -s CYBER_DEFAULT 
I1123 20:20:31.816969 14684 global_data.cc:153] []host ip: 192.168.1.6
I1123 20:20:31.818890 14684 module_argument.cc:82] []binary_name_ is mainboard, process_group_ is trafficlight, has 4 dag conf
I1123 20:20:31.818898 14684 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_region_proposal/dag/traffic_light_region_proposal.dag
I1123 20:20:31.818902 14684 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_detection/dag/traffic_light_detection.dag
I1123 20:20:31.818925 14684 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_recognition/dag/traffic_light_recognition.dag
I1123 20:20:31.818928 14684 module_argument.cc:85] []dag_conf: /apollo/modules/perception/traffic_light_tracking/dag/traffic_light_tracking.dag
I1123 20:20:31.820008 14703 third_party/var/bvar/variable.cpp:792] Write all bvar to /apollo/dumps/traffic_light_region_proposal.dag_traffic_light_detection.dag_traffic_light_recognition.dag_traffic_light_tracking.dag every 10 seconds.
E1123 20:20:34.242411 14684 sensor_manager.cc:79] [mainboard]Failed to load camera intrinsic:/apollo/modules/perception/data/params/onsemi_obstacle_intrinsics.yaml
E1123 20:20:34.242430 14684 sensor_manager.cc:93] [mainboard]Failed to add sensor_info: onsemi_obstacle
F1123 20:20:34.242449 14684 sensor_manager.cc:37] Check failed: this->Init() == true (0 vs. 1) 
*** Check failure stack trace: ***
    @     0x7f21eb20f67d  google::LogMessage::Fail()
    @     0x7f21eb211b3a  google::LogMessage::SendToLog()
    @     0x7f21eb20f08d  google::LogMessage::Flush()
    @     0x7f21eb212ae9  google::LogMessageFatal::~LogMessageFatal()
    @     0x7f21d3e286f0  apollo::perception::algorithm::SensorManager::SensorManager()
    @     0x7f21dfcea68b  _ZZN6apollo10perception9algorithm13SensorManager8InstanceEbENKUlvE_clEv
    @     0x7f21dfd020b0  _ZSt13__invoke_implIvZN6apollo10perception9algorithm13SensorManager8InstanceEbEUlvE_JEET_St14__invoke_otherOT0_DpOT1_
    @     0x7f21dfcf8901  _ZSt8__invokeIZN6apollo10perception9algorithm13SensorManager8InstanceEbEUlvE_JEENSt15__invoke_resultIT_JDpT0_EE4typeEOS6_DpOS7_
    @     0x7f21dfcf0da5  _ZZSt9call_onceIZN6apollo10perception9algorithm13SensorManager8InstanceEbEUlvE_JEEvRSt9once_flagOT_DpOT0_ENKUlvE_clEv
    @     0x7f21dfcf0ddd  _ZZSt9call_onceIZN6apollo10perception9algorithm13SensorManager8InstanceEbEUlvE_JEEvRSt9once_flagOT_DpOT0_ENKUlvE0_clEv
    @     0x7f21dfcf0dee  _ZZSt9call_onceIZN6apollo10perception9algorithm13SensorManager8InstanceEbEUlvE_JEEvRSt9once_flagOT_DpOT0_ENUlvE0_4_FUNEv
    @     0x7f21eadd9907  __pthread_once_slow
    @     0x7f21dfcd60d5  __gthread_once()
    @     0x7f21dfcf0e74  _ZSt9call_onceIZN6apollo10perception9algorithm13SensorManager8InstanceEbEUlvE_JEEvRSt9once_flagOT_DpOT0_
    @     0x7f21dfcea700  apollo::perception::algorithm::SensorManager::Instance()
    @     0x7f21df5ecee1  apollo::perception::trafficlight::MultiCamerasProjection::Init()
    @     0x7f21df610721  apollo::perception::trafficlight::TLPreprocessor::Init()
    @     0x7f21dfcd7aa7  apollo::perception::trafficlight::TrafficLightsPerceptionComponent::InitAlgorithmPlugin()
    @     0x7f21dfcd6fcb  apollo::perception::trafficlight::TrafficLightsPerceptionComponent::Init()
    @     0x55c2efc6b9a4  apollo::cyber::Component<>::Initialize()
    @     0x55c2efc6fea6  apollo::cyber::mainboard::ModuleController::LoadModule()
    @     0x55c2efc70314  apollo::cyber::mainboard::ModuleController::LoadModule()
    @     0x55c2efc6f819  apollo::cyber::mainboard::ModuleController::LoadAll()
    @     0x55c2efc6bd7c  apollo::cyber::mainboard::ModuleController::Init()
    @     0x55c2efc6a598  main
    @     0x7f21ea0bbc87  __libc_start_main
    @     0x55c2efc6a0aa  _start
    @              (nil)  (unknown)
[cyber_launch_14677] ERROR Process [trafficlight] has died [pid 14684, exit code -6, cmd mainboard -d /apollo/modules/perception/traffic_light_region_proposal/dag/traffic_light_region_proposal.dag -d /apollo/modules/perception/traffic_light_detection/dag/traffic_light_detection.dag -d /apollo/modules/perception/traffic_light_recognition/dag/traffic_light_recognition.dag -d /apollo/modules/perception/traffic_light_tracking/dag/traffic_light_tracking.dag -p trafficlight -s CYBER_DEFAULT].
[cyber_launch_14677] INFO All processes has died.
[cyber_launch_14677] INFO Cyber exit.
[cyber_launch_14677] INFO All processes have been stopped.

So, my questions would be the following:

  • Did I do something wrong during the project installation that could have caused an error when trying to run the traffic cameras module?
  • If it's not an issue with the installation itself, could it be a configuration problem, or perhaps something related to the selection of vehicles/map/setup mode?
  • If the problem is not related to any of the above, how can I resolve it?
  • Lastly, does it make sense, knowing that my idea is to run the traffic cameras process on my own video, to go through all of this? If so, how can I use my own video without having to go through all the hardware installation steps?

Output from apollo.sh config: Screenshot from 2024-11-23 19-43-44

Output from cyber_launch start /apollo/modules/transform/launch/static_transform.launch: Screenshot from 2024-11-23 19-44-30

Output from cyber_launch start /apollo/modules/perception/launch/perception_trafficlight.launch with car Mkz Lgsvl 321: Screenshot from 2024-11-23 19-44-07

Output from cyber_launch start /apollo/modules/perception/launch/perception_trafficlight.launch with car Kitti 140: Screenshot from 2024-11-23 20-20-17

Output from cyber_launch start /apollo/modules/perception/launch/perception_trafficlight.launch with car Mkz Example: Screenshot from 2024-11-23 20-20-43

cirobrignolo avatar Nov 23 '24 23:11 cirobrignolo

However, the process never completed, and the output seemed to stall. The output was:

When you launch a node, it keeps running and waiting for input messages. So it is normal in my opinion.

The output contains many errors related to lane width. Skipping repetitive lines for brevity, the process eventually failed with:

Seems that lane issues are related to HDMap

E1123 18:31:49.268774  9088 hdmap_common.cc:158] [map]lane[id = id: "7657a_1_-1"]. sampled_right_width_[0.997895] is too small. It should be larger than half vehicle width[1.05].

The errors are because it cannot find the pre-trained model as shown by the error messages

E1123 18:31:54.277089  9088 rt_utils.cc:48] [mainboard]cannot open file /apollo/modules/perception/data/models/tl_detection_caffe/baidu_iter_140000.caffemodel

You should locate this file in your version, simply search the file name.

In my opinion, you do not need to launch the entire project for traffic light. You only need several modules, like HDMap and Traffic Light, and Localization. You do not need dreamview or whatever.

If you have your own video, you have to have the corresponding map and localization to provide the ROIs, Check this https://github.com/ApolloAuto/apollo/blob/master/docs/06_Perception/traffic_light.md Apollo traffic light detection cannot work for the entire picture. It has to have an initial estimation of the traffic light location in the picture. Then it detects traffic lights from these small regions (i.e., ROIs). In production, this information is from localization and HDMap.

I have replicated the Apollo traffic light detection and recoginition pipeline in python. It can process pictures with specified initial estimations. Check about this https://github.com/boyang9602/TLR-replicate

boyang9602 avatar Nov 25 '24 17:11 boyang9602

@boyang9602 I was looking at the Git project you referenced, but I have some doubts about the project itself. I understand that it works by taking an image and some bounding boxes you pass as parameters, and it provides the pipeline's output that would occur in the Apollo project. My question here is: how do you determine which bounding boxes to pass as parameters? Also, there's something I wanted to test based on the video, which is step 4 of the pipeline, the "reviser." This step could determine the current traffic light information based on the prior information it had. Is that possible here? Lastly, you mentioned that certain map and location configurations were necessary. Are those required here as well?

cirobrignolo avatar Jan 14 '25 23:01 cirobrignolo

how do you determine which bounding boxes to pass as parameters?

In Apollo, it is determined using a camera model to project the 3D traffic lights into 2D pictures. It requires the 3D model of the environment (HDMap), Localization, and camera information. You can check the pre-process step described in https://github.com/ApolloAuto/apollo/blob/master/docs/06_Perception/traffic_light.md#pre-process. If you only have a video without the 3D model of the environment and the location of your car, you cannot do this step.

However, you can simulate this step by adding some transformation of the ground truth boxes (e.g., shift, resize, etc), i.e., making up some projection bounding boxes using ground truth information, as if they are calculated using the HDMap and localization.

This step could determine the current traffic light information based on the prior information it had. Is that possible here?

My replication does not include the reviser. However, it is simple as described by the doc

If the Reviser receives a definitive status such as red or green, the Reviser saves and outputs the status directly. If the received status is black or unknown, the Reviser looks up the saved map. When the status of this light is certain for a period of time, the Reviser outputs this saved status. Otherwise, the status of black or unknown is sent as output.
Because of the time sequence, yellow only exists after green and before red. Any yellow after red is reset to red for the sake of safety until green displays.

Lastly, you mentioned that certain map and location configurations were necessary. Are those required here as well?

Please refer to the reply for the first question.

boyang9602 avatar Jan 15 '25 02:01 boyang9602