Error when I run "run_evaluation.sh"

[mehdi-maleki]

Hi, Thanks for the interesting work and well documentation!

I follow the instructions provided in the README file to run the Autopilot but I face the following errors:

1. I got this error (pic below) when I tried to run "run_evaluation.sh", I solved this problem by creating a directory named "carla_results". Is this the right solution?

2. Now, I'm getting another error (pic below), what can be the problem?

Any tips would be helpful.

Thanks in advance!

[kashyap7x]

1. CHECKPOINT_ENDPOINT in run_evaluation.sh is the directory where the results file will be written, you can set this to any valid file path. Creating the carla_results folder is a valid solution.
2. You seem to be using an incompatible CARLA version. We use 0.9.10.1, see the install instructions here. This includes the instructions to install additional maps that must be included to run our experiments. Once you have the right CARLA version and additional maps, please confirm that your CARLA_ROOT is set correctly, and the carla-0.9.10-py3.7-linux-x86_64.egg file is added to the PYTHONPATH.

[mehdi-maleki]

Thanks for the reply. It's now working. I'm also using your other project (Transfuser), so I thought the same Carla would work but forgot to change the root path in this project.

[mehdi-maleki]

I have a few more questions if you could answer:

1. When I compare the speed of the simulation run between Neat and Transfuser, I see that there is a noticeable difference in cars speed (the car in the Neat project drives faster ). However, when I check the speed limits in both of the projects are the same as {0, 4, 7} m/s. What is the reason for that? and how can I speed up the simulation run in the Transfuser project?

2. In which file the simulation configuration (e.g., FPS) is defined? I'm wondering if I can change them as I want?!

Thanks!

[kashyap7x]

1. If you are referring to the simulation speed of the autopilot, there should be no differences between NEAT and Transfuser. Please verify that you compared the 2 simulations with the same environment variable flags. For example, if you are saving the data (via SAVE_PATH) in one case, but not saving it in the other, this leads to a significant difference in simulation time.
2. We use the official simulation frequency of the CARLA leaderboard (20 FPS) - we are not allowed to edit this for the CARLA challenge, so it is not in any config file. However, you can change it here: https://github.com/autonomousvision/neat/blob/686f4a0b5b5bf20c99f323e9542f5b68808df2de/leaderboard/leaderboard/leaderboard_evaluator.py#L67 Note that CARLA's dynamics fail if the simulation is made slower than 10 FPS - this is the minimum value you can set.