Run a list of multiple scenarios with different start locations for automated testing

[Leon-Adler]

Hello together,

I would like to have a scenario executed automatically several times in a row (such as FollowLeadingVehicle) and record it for automated tests. I would also like to reset the starting position of the vehicle in front of me in a slightly different way.

In fact, this matter has already been implemented and resolved.

The only hurdle that still has to be overcome is that between the scenarios after their termination I still have to kill the process by "CTRL + C" before the next scenario starts.

Hence the question what could be the reason for this and if there is a better approach?

The last error message before the process is killed by "CTRL + C" is as follows:

Not all scenario tests were successful
Destroying ego vehicle 439
ERROR: failed to destroy actor 439 : unable to destroy actor: not found 
No more scenarios .... Exiting
Scenario ended -- Terminating
WARNING: attempting to destroy an actor that is already dead: Actor 439 (vehicle.lincoln.mkz2017) 

In the following the two Bash scripts are presented. The first script creates a list with three FollowLeadingVehicle_1 entries which are executed one after the other with the command "bash scenario_runner.sh $scenario_name true $scenario $location". This command executes the second bash script, which basically only executes (1) ScenarioRunner, (2) Manual_Control, and (3) Carla Recorder.

Bash script 1:

#!/bin/bash
#listScenarios := list of test scenarios
#scenario_name := a scenario from the list "listScenarios"
#scenario := Count Variable for the scenarios if you want to the test the same scenario 2 times in the same execution
#location := an additional argument from command line to modify the location

scenario=1
location=$1

#Define the list of the scenarios you want to test
listScenarios="FollowLeadingVehicle_1 FollowLeadingVehicle_1 FollowLeadingVehicle_1"

#Iterate over listScenarios
for scenario_name in $listScenarios; do

    #Execute the scenario_runner.sh with the 4 arguments (name, true/false, scenario count, location)
    bash scenario_runner.sh $scenario_name true $scenario $location

    #Kill the scenario process
    #STRG + C

    #Increment scenario counter
    let "scenario++"

    #Decrement start location off the first vehicle
    ((location=location-10))

done

Bash script 2:

#!/bin/bash

#1) Start the ScenarioRunner
echo "The $1 scenario will be executed"
python scenario_runner.py --scenario $1 --output --reloadWorld --modify $4 &

#2) Start the manual control
echo "The manual control will be executed"
python manual_control.py &

#3) Optional Record the Scenario
if $2; then
    echo "The file will be recorded"
    python /home/carlaws19/CARLA_0.9.9/PythonAPI/examples/start_recording.py -f "$1_$3_$4.log" -n "0"
else
    echo "The file will not be recorded"
fi

Further approaches and ideas for test automation can be found under the following link: https://github.com/ll7/scenario_runner/blob/master/Docs/Semi_automated_tests.md

[arjunsk07]

@Leon-Adler Did you were able to solve the hurdle that you mentioned here? In my case also, I need to maually do the "CTRL + C" to exit the scenario and to run another one. Did you got a solution to this?

[wnklmx]

Hi @arjunsk07, I set up a subscription to this issue a long ago and wanted to add my solution. Unfortunately, I solved my specific problem a long time ago and I'm not 100 % sure if what I did will also help with your problem.

For me the scenario_runner did not exit since the vehicle never reached the last waypoint. This may only apply to route_scenarios, here reaching the last waypoint is usually an exit condition. Hence, I created two routes: one with all waypoints that the vehicle is supplied with. And one route that misses the last waypoint that is used for the exit condition. So if the vehicle reaches the penultimate waypoint (which it does), the scenario exit condition is achieved and the scenario_runner execution terminates.

You can finde these edits in https://github.com/wnklmx/scenario_runner/blob/master/srunner/scenarios/route_scenario.py following the varialble gps_route_without_last_waypoint.

I hope that this helps you to figure out the issue.

[starrye]

If you're using OpenScenario. Try setting the starting position as a variable, and then start start_scenario multiple times, passing in the position argument.