An error occurred while enabling the op_trajectory_evaluator with autoware.ai

[dean1314]

Hello. I‘m trying to use op_trajectory_evaluator to generate local_weighted_trajectory paths on autoware demo map, but an error occured while enabling op_trajectory_evaluator. Below are the steps the reproduce the issue:

1. Execute runtime manager.
2. Go to "Map" tab and enable "Point Cloud" and "Vector Map" and "TF" - The corresponding file provided by autoware demo.
3. Go to "Setup" tab and enable "TF" and "Vehicle Model"
4. Go to "Computing" tab and enable "vel_pose_connect" - Make sure selected the "Simulation Mode", enable lidar_kf_contour_track, enable op_global_planner - Make sure selected the "Lane Changing", "Replaning" and "Rviz Goals", Velocities is set to "Autoware".
5. Open Rviz by clicking the "RViz" button on Runtime Manager.
6. selected the start position by using "2D Pose Estimate" tool and Goal position by using "2D Nav Goal" tool - Make sure the global path having lane changing like the following picture.
7. continue to enable some model like the following picture
8. Publising current pose by using "2D Pose Estimate" tool
9. enable op_trajectory_evaluator and the error occured in terminal as the following picture

but the error were not occur when the global path don't have lane changing.

Thank you very much in advance for your help!

[hatem-darweesh]

This issue was fixed in my repository. kindly check OpenPlanner working group for more details. https://discourse.ros.org/t/openplanner-2-0-requirements-design-and-implementation/11985/4

[marcusvinicius178]

Hi @hatem-darweesh I tried to use OpenPlanner With Autoware.Ai master branch...

I have executed all the steps required on documentation. I am simulating with SVL simulator. Unfortunately the OP_Global planner is not drawing/generating the trajectory. I have debuged using RQT from runtime manager and using rostopic echo. I have echoed all the required subscription topics and I have realized all of them have data, with EXCEPTION of vector_map info! Anything is published in this topic.

However I checked and the vector map is provided in my launch file. I also have loaded it in runtime manager setup tab. The LGSVL provided this vector map: https://github.com/lgsvl/autoware-data/blob/master/CubeTown/my_launch/my_map.launch

For this reason I don't understand why OP global planner is not generating the trajectory. I am on cell phone now but I will send later the rosgraph from RQT and videos displaying this attempt.

In addition I see a lot of videos on internet with Toyota2 city and a kind of AUTOWARE internal simulator where the planners work easily...Do you have this map, is it available?

Please take a look here:https://github.com/Autoware-AI/autoware.ai/issues/2387

Do you know what are the problems to distort the global planners? Why the local planners are not avoiding the collision? I will be very grateful if you could send me some advices on this issue too

Finally I just would like to use Autoware.Ai to avoid an obstacle with LGSVL simulator. Is this possible? Until know any planner was able to do that, just the freespace planner, but with a very bad motion performance.

[hatem-darweesh]

The updated OpenPlannet 2.5 is integrated with Autoware.AI master branch. The integration is at my own fork https://github.com/hatem-darweesh (common, core_planning, core_perception) check branch "openplanner.1.15" you can find tutorials about how to install and use it with different Autoware.AI releases. and more information could be found in the documents and videos here https://discourse.ros.org/t/the-start-of-autoware-openplanner-development-wokring-group/19241/6?u=hatem

[marcusvinicius178]

Hi Mr. Darweesh Thanks to provide me this detailed information!

Before testing your planner I have a couple of doubts if you have a time please:

As far as I know Autoware.Ai reached just the 1.14 release tag. I have not found the 1.15 as you said when I clicked over Tags.

In addition I have seen you have a tutorial for Carla simulator. But I am using the LGSVL simulator, with these maps: https://github.com/lgsvl/autoware-data

Does OpenPlanner also work for these maps? Because I have checked the Global planner OpenPlanner core node against the core node of way_planner global planner. The structure is similar, with exception the OpenPlanner gets more complex vector map info (as curbs, way_areas,cross_walks, intersect,etc.... while the way_planner needs to be fed just with vector maps pole, lane, lines and points). Maybe this be the reason why I get no data flow from vector_map_info topic (required subscription off OpenPlanner)....because LGSVL vector maps just provides these "basic" vector map info...therefore the OpenPlanner does not work with LG maps? Maybe Carla offers all these Vector Map info....?

When you say to get your fork repo, you mean I should download the modules: core-planning, core-perception and common, and replace them inside the Autoware folder inside the docker image created from Autoware.Ai master repos? Or must I compile them on Shared_dir? That's complicated because when I exit the docker image everything is erased, unless I copy your repos to Dockerfile maybe. And if I build them on Shared_dir the launch files of runtime manager does not find them passing the path on includes... What's the best solution approach for that?

Or do you mean that s just command a "git checkout" inside docker terminal and the branche option "1.15 open planner" will appear?

A last question is: The Global Open Planner is the same as Global Way Planner? (Same behavior)? The Op Local Planner is the same as Dp planner?

I ask because I have seen this picture on YouTube:

if they be the same, I believe the car will keep not avoiding collision, considering I have done a lot of tests with these planners and the local planners A*, dp or even Lattice could not distort/Remake the Global Planner path when an obstacle was found by velodyne/occupancy grid map. Take a look: https://github.com/Autoware-AI/autoware.ai/issues/2387

Have you a tutorial or video of collision avoidance using this planner with SVL simulator?

Thanks in advance.

[hatem-darweesh]

As far as I know Autoware.Ai reached just the 1.14 release tag. I have not found the 1.15 as you said when I clicked over Tags.

We refer to Autoware.AI master as 1.15

In addition I have seen you have a tutorial for Carla simulator. But I am using the LGSVL simulator, with these maps: https://github.com/lgsvl/autoware-data

It should work the same way, unless LGSVL map has an issue, if these maps can be loaded using assuremaptools then it will work with openplanner

When you say to get your fork repo, you mean I should download the modules: core-planning, core-perception and common, and replace them inside the Autoware folder inside the docker image created from Autoware.Ai master repos? Or must I compile them on Shared_dir?

Clone everything first as the instruction in the tutorial video and documents, then build the docker image it will include the latest.

A last question is: The Global Open Planner is the same as Global Way Planner? (Same behavior)? The Op Local Planner is the same as Dp planner?

Please don't use way_planner and dp_planner, they are the old openplanner 1.0 nodes. they are left their for compatibility reasons only,. Use op_global_planner + op_local_planner + lidar_kf_contour_tracker

[marcusvinicius178]

@hatem-darweesh I was following this tutorial https://www.youtube.com/watch?v=7Dg1WT9n1Po to install autoware_ai pkgs from source. However I am having a bunch of errors to successfully compile the pkgs. I believe because these pkgs have complex dependencies as written here (https://gitlab.com/autowarefoundation/autoware.ai/autoware/-/wikis/Source-Build):

I running the Autoware.AI inside docker: https://github.com/Autoware-AI/autoware.ai/wiki/Generic-x86-Docker CASE 3 of the link above. I have built the master (1.15.0) on my docker image.

In this way your tutorial (From source does not work for me) and is not possible then, to dowload your repo inside this docker image and afterwards switch to your branche:

git checkout openplanner1.15:

Do you have another specific tutorial to build the Docker Image within your repos: common, perpection and core-planning included with other Autoware master repos modules?

I have tried to work around the DockerFile and set to it get the autoware.ai.repos from a local folder and not from wget. In addition I have changed the dowload of the modules: common, core_planning, and perception modules from master version to open_planner 1.15 branche:

and then I have build the docker image with:

./build.sh ./run.sh -t local

However error raises and then a standard local-cuda-melodic image is created instead my customized with the modules from Hatem-Darshee github repos.

How could I get these forked repo modules (common, planning and percpetion) from Hatem and the other modules from master?

Please if you have already worked on the DockerFile to create this customized Autoware-OpenPlanner image could you send me? otherwise unfortunately I will not be able to run your planner in my research as I am using a shared computer and cannot brake the system trying the installation from source.

Thanks in advance

[marcusvinicius178]

As I still couldn't test the OpenPlanner 2.5 due Docker/Source issues installation I tried the 1.5 OpenPlanner (as your pdf material said, it is included in Autoware.AI masterfile:///home/autoware-auto-ros1/Downloads/OpenPlanner_Presentation.pdf):

I have debugged to understand why the global trajectory was not drawed in rviz. You can take a look if you wish on my debug video to help me if possible (sorry for the slow motion video, but I rent a cloud machine as my notebook is not powerful enough to run the simulations with Apollo/Autoware):

https://youtu.be/D6hrEm9DYGM

Well you can also check my Rosgraph from RQT which I have saved for you take a look:

Comparing with the rosgraph from your pdf and reading as OpenPlanner works (also with much ROS experience that I have and Autoware) everything seems good with exception that I have echoed the topics and no messages available at topic lane_waypoints_array. Then I have echoed all the vector_maps info that feed the op_global_planner node....and for my misfortune more than half of these topics does not send information to the global planner node. As I was thinking the LGSVL vector maps info are not enough. Just take a look what I have on these maps:

I do not have for example the vector map info from: -curb -way_area -area -road_edge -cross_road -cross_walk

That's why I have said you in the beginning that the Op global planner core node was pretty similar (in structure) to the way_planner. The difference is that the Op global planner requires/asks for more vector_map info, therefore I believe that when these info does not arrive to the node, it cannot produce the global planner to rviz and neither the local ones...

What is your point of view? Unfortunately I think I will not be able to use your planner with LGSVL simulator. But at least I can give this explanation om my master thesis...it is a pitty because I would like to compare a planner in Autoware vs Apollo. However in Autoware any global+local planner was able to avoid collision. At least with LGSVL simulator...

**Any idea for my specific case (LGSVL-OpenPlanner 1.15 or 2.5 version) Mr. Darweesh? @hatem-darweesh **

Thanks for your attention and congratulations for your job, meeting and open planner project! I have followed your hard work during these 4 years! Pretty nice!

[hatem-darweesh]

1- Install on a clean version of Ubuntu 18.04 2- Don't build with CUDA (It is difficult because the CUDA versions) 3- Don't use docker (I never used Autoware inside docker)

If you follow these three rules, most probably your issues will disappear.

[marcusvinicius178]

Ok but I think in the end I will have the same issue with the lgsvl vector maps lack of info to feed the Op global planner node, as I displayed you above right?

The AUTOWARE vector maps (with those CSV I have showed you) + Autoware pcd maps are automatically used by the Assure maps when using global planner in runtime manager or must I run this tool (Assure maps) externally? I mean I should take the CSV vector map info from Lgsvl maps + the pcd map and feed as input to the Assure Maps tool and this tool will convert for some other kind of map as output of this process?

[marcusvinicius178]

The OpenPlanner 1.5 is already integrated with Autoware.Ai Master (1.15) right? I have read in your pdf presentation that it is enough to collision avoidance. Thus if 1.5 works for me it will fit my needs and I won't need the 2.5 Open Planner and the complicated dependencies installation on my Ubuntu (eigen, etc) that will probably crash the system (docker is very important to avoid this). Is it necessary to git checkout to other branche to use the 1.5 Open Planner also or using master for all modules is enough? Unfortunately I cannot start from a clean Ubuntu 18.04 I have a lot of other ROS projects and other tests with Apollo, MoveIt and so on running on my Linux and also in virtual machine. Erase then or install a new Linux alongside them it is too risk, because I have not created another partition (just one for Linux and other for windows). If I do this I will lose everything probably and a backup would take too much time to do and reinstall everything again after all.... that's why I prefer to work with docker, it is much better.

[hatem-darweesh]

OpenPlanner is integrated with Autoware.AI master (1.15) the same as 1.13, through my repository. not into the official Autoware repo. This is the way I have been working and testing. please ask someone professional in docker to solve your docker issues.

[marcusvinicius178]

ok I understood. I really would like to test it, maybe after end my master thesis (unfortunately I have just 1 month to finish it and cannot take the risk to install a new ubuntu in my machine, I will try maybe to rent another pc on cloud. However there are a lot of dependencies to be dowloaded, right? In addition it seems that LGSVL csv vector maps does not fit all the information that the global planner needs, as RQT returned me. Therefore I am not sure if it is going to work with SVL simulator.

Anyway thanks very much for your kind feedback. Wish you success with your planner.

[marcusvinicius178]

Hi Mr. hatesh @hatem-darweesh I have finally modified my Dockerfile to accept other repos, not just from Autoware....

Therefore I could test the Open Planner 2.5 in Docker environment. The global path was drawed, but the car did not move and I have followed these instructions from this video: https://www.youtube.com/watch?v=BS5nLtBsXPE

Afterwards I tried to launch all the modules written in your pdf sketches: file:///home/autoware-auto-ros1/Downloads/OpenPlanner_Presentation%20(1).pdf

You can check I have activated all of them:

As you can see above on terminal I have also launched the op_waypoint_follower by command line, as I did not see this option available on runtime_manager interface.

Also you can check these attempts on my videos:

https://youtu.be/b_NBgBwkRsI

https://youtu.be/GrNl4atR4Vc

Below is the **rosgraph output from RQT instrospection nodes/topics **:

As you can check in videos or through the picture below, the car is receiving /vehicle_cmd

However it is still not moving, it does not follow the path

I have also echoed the topics:

-global_waypoints_rviz -lane_waypoints_rviz /pmap_stat -steer_cmd -accel_cmd -brake_cmd

All of them contain messages and are active. I thought the pure_pursuit + twist_filter nodes would be enough to enable the car following the path and move...

Could you help me please to understand what can be wrong? I mean what is missing to be launched or if I need to configure some specific parameter in a launch file to enable the car move with the simulator?

Thanks in advance

[marcusvinicius178]

@hatem-darweesh Just figured out how to enable the car move. The car however had a confuse behavior...I mean it has followed previous global waypoints that I have sent in rviz. Does it kind of cumulate goal points even when you change the goal spot? Doesn't it forget the goal points when you change in rviz?

Also I have just activated the OP global planner and all op_local planner launch files related to local planner. I did not launch this time: -any autoware_connector launch files -neither the Open Planner Simulator launch files

• any op_perception_simulator
• any op-utilities launch files.
• I have not launched the wf_simulator from Waypoint follower icon too.

Are some of them necessary to be launched to improve the vehicle,planner behavior? Or have not they any function with SVL simulator? I am not sure if these launch files are related to the use of other simulator...

When I simulated the vehicle got some shortcuts path. I mean OP planner did not identife or respected the curbs. Probably because the provided maps from LGSVL does not contain a curb info vector map just other info...is this the reason?

Other issue is that the Ego vehicle identified just 70% I would say from the obstacle and therefore after overtake part of it, it returns to the lane too early and collides with the front part of the obstacle. I don't know if the lidar_contour_track is enough to identify the obstacle properly? Must I launch other file? I know other planners work with occupancy grid map to avoid collision. Does your planner take information from occupancy grid too or not? How could I improve obstacle detection/avoidance?

Thanks in advance. And sorry to kind of criticize the non-use of Docker for your branches I did not wish to be rude...just wished to have less compatibility issues...

I know other guys would like to solve the car motion problem and Docker...so I am available to create a new issue in GitHub (because this is not mine) to comment about these topics. If you wish too.

[marcusvinicius178]

Now the planner is working 100%!!! It was just another issue with track lidar contour detector was subscribing to wrong topic in source code. Now everything is working! Do you have a paper to cite it? I would like to cite in my thesis. Thanks!

[hatem-darweesh]

@marcusvinicius178 That is great, I am happy to know. You can cite these papers: 1- https://www.fujipress.jp/jrm/rb/robot002900040668/ 2- https://www.jstage.jst.go.jp/article/jsaeijae/10/4/10_20194117/_article/-char/ja/

[marcusvinicius178]

Hi again @hatem-darweesh could you help me please with yout path planner interpretation?

I need to plot the GLOBAL path waypoints vs LOCAL path waypoints in the same graph.

I have extracted a bag file and I am plotting the data from a csv. As you are aware there are a lot of topics published.

But I just need to understand which one returns me the waypoints of the Global Path (In BLUE):

And which topic returns me the Local Path (in Green):

As you can check I have extracted all the topics present in the ROSBAG:

I checked in the source code that the Global Planner is produced by the op_global_planner node and is sent to the topic --> /lane_waypoints_array --> op_trajectory_generator node and also to --> dp_planner node--> /local_trajectories topic--> op_trajectory_evaluator node--> /local_weighted_trajectories and /local_trajectory_cost topics --> op_behavior_selector node --> /final_waypoints --> control and from op_behavior_selector node --> /op_global_planner -->lane_waypoints_array --> op_trajectory_generator node again...

These conclusions were done also from this instrospection of RQT:

The csv files produced by the topics that I think that returns the Global and Local Trajectory are very HUGE! And I cannot understand which of them have the data of the Global path and of the Local path. From the introspections it seems that the topics contain data of both together....how could I separate to visualize which data is from global and which is from local?

The topic that the open Global planner node publishes data are:

1-Op_global_planner: A) /global_waypoints_rviz (the data is strange, just for visualization I imagine) B) /lane_waypoints_array

The topics that the Local planner nodes publishes are :

1- op_behavior_selector: A) /op_global_replan (However my bag did not record this topic for some reason that I do not know) B) / behavior_state (However this topic does not publish data to any topic as you can check in rosgraph) C) /final_Waypoints

2 -op_trajectory_evaluator: A) /local_weighted_trajectories B) /local_trajectory_cost C) /safety_border (does not publish data to any topic)

3-op_trajectory_generator: A) /local_trajectories

At the end I wish to achieve something like this:

Here there is a short video of the data that is making me confusion during the plot:

https://youtu.be/uHjbZ2RJwsM

I imagine you are the best person that could assist me to achieve that plot, if possible, please let me know which topic and which columns, waypoints, etc is the global path and which topic is the local path...

Thanks in advance.

[bilibilicharger]

Hi， @hatem-darweesh ， I also encountered this problem. I now install autoware 1.14. Do I need to uninstall autoware if I want to install openplanner?

[hatem-darweesh]

@bilibilicharger follow these instruction to use OpenPlanner 2.0+ either with Autoware.AI 1.13 or latest master.

[bilibilicharger]

@hatem-darweesh Thank you for reminding me. I'll follow the guidance.