--visualize_world not displaying complete visualisation

[JCSadeghi]

I am having trouble using the --visualize_world option as described here: https://pylot.readthedocs.io/en/stable/visualization.html

However I do not see the colourful map shown on the documentation - instead I see the following:

In addition my command line shows many errors such as the following:

Traceback (most recent call last):
  File "/home/erdos/.local/lib/python3.8/site-packages/erdos/streams.py", line 188, in send
    return self._py_write_stream.send(internal_msg)
WriteStreamError.TimestampError: Error sending message on a7015556-7321-9352-b74e-a6694460cab1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/erdos/workspace/pylot/pylot/simulation/carla_operator.py", line 251, in send_actor_data
    self.__send_hero_vehicle_data(self.pose_stream, timestamp)
  File "/home/erdos/workspace/pylot/pylot/simulation/carla_operator.py", line 366, in __send_hero_vehicle_data
    stream.send(erdos.Message(timestamp, pose))
  File "/home/erdos/.local/lib/python3.8/site-packages/erdos/streams.py", line 190, in send
    raise Exception("Exception on stream {} ({})".format(
Exception: Exception on stream pose_stream (a7015556-7321-9352-b74e-a6694460cab1)

The other visualisations work correctly for me (e.g. --visualize_detected_obstacles) If you can reproduce the problem, please could you share a known commit hash when this feature was working?

Here is my config, please let me know if you see any mistakes:

--scenario_runner
--simulator_mode=synchronous
--random_seed=1337
###### Perception config #####
--perfect_obstacle_detection
--nosimulator_obstacle_detection
--simulator_traffic_light_detection
--obstacle_tracking
--min_matching_iou=0.2
--tracker_type=sort
--obstacle_track_max_age=3
--ignore_obstacles_with_short_history=3
--dynamic_obstacle_distance_threshold=50
--static_obstacle_distance_threshold=70
######### Prediction config #########
--prediction
--prediction_type=linear
--prediction_num_past_steps=5
--prediction_num_future_steps=30
######### Planning config #########
--planning_type=frenet_optimal_trajectory
--target_speed=10
--max_speed=20
--d_road_w=0.3
--d_t_s=0.25
--dt=0.1
--obstacle_clearance_fot=0.7
--obstacle_radius=1.5
--maxt=6.0
--mint=4.0
--ko=100
--max_curvature=10.0
--max_accel=7.0
--max_road_width_l=5.0
--max_road_width_r=0.85
--num_waypoints_ahead=50
--goal_location=47.73, 327.07, 0.5
###### Control config #####
--control=pid
--stop_for_vehicles=False
--stop_for_people=False
--stop_for_traffic_lights=False
--steer_gain=1.0
######### Other config #########
--visualize_world

[Morphlng]

Hello! I'm facing the same problem and even more, --visualize_detected_obstacles doesn't work for me too.

Are you using the Carla integrated inside the docker container? I'm using a outside Carla 0.9.13 and when toggle on --visualize_detected_obstacles, this line of code always raise an exception:

https://github.com/erdos-project/pylot/blob/a71ae927328388dc44acc784662bf32a99f273f0/pylot/perception/detection/obstacle.py#L158-L165

I investigated how exactly the corners are calculated and tried to replace this raising with the calculation process. However, the result doesn't seem right. I also tried to simply modify this line:

corners = self.bounding_box.to_camera_view( 
   ego_transform, frame.camera_setup.get_extrinsic_matrix(), 
   frame.camera_setup.get_intrinsic_matrix()) 

This will show some point, yet not in the place of the vehicles.

[Morphlng]

Here is my config, please let me know if you see any mistakes:

######### Other config #########
--visualize_world

I've solved the problem of --visualize_world not working properly. It's because you have to pass also --visualize_prediction to show the semantic information. The code logic is written in here:

https://github.com/erdos-project/pylot/blob/a71ae927328388dc44acc784662bf32a99f273f0/pylot/debug/visualizer_operator.py#L365-L376

To have that message, you must create the stream:

https://github.com/erdos-project/pylot/blob/a71ae927328388dc44acc784662bf32a99f273f0/pylot/operator_creator.py#L888-L890