Inconsistent scenario types returned
I might have fundamentally misunderstood the concept of "scenario types", but the following behavior seems odd to me.
# filter by 'starting_unprotected_cross_turn'
f1 = ScenarioFilter(
scenario_types=['starting_unprotected_cross_turn'],
scenario_tokens=None,
log_names=None,
map_names=None,
num_scenarios_per_type=None,
limit_total_scenarios=None,
timestamp_threshold_s=None,
ego_displacement_minimum_m=None,
expand_scenarios=False,
remove_invalid_goals=False,
shuffle=False,
)
scenarios = builder.get_scenarios(scenario_filter=f1, worker=SingleMachineParallelExecutor())
s = [s for s in scenarios if s.token == '454B09876AC457D2'.lower()][0]
print(s.token) # -> 454b09876ac457d2
print(s.scenario_type) # -> starting_unprotected_cross_turn
However:
# filter by token
f1 = ScenarioFilter(
scenario_types=None,
scenario_tokens=[t.lower() for t in ['454B09876AC457D2']],
log_names=None,
map_names=None,
num_scenarios_per_type=None,
limit_total_scenarios=None,
timestamp_threshold_s=None,
ego_displacement_minimum_m=None,
expand_scenarios=False,
remove_invalid_goals=False,
shuffle=False,
)
scenarios = builder.get_scenarios(scenario_filter=f1, worker=SingleMachineParallelExecutor())
s = [s for s in scenarios if s.token == '454B09876AC457D2'.lower()][0]
print(s.token) # -> 454b09876ac457d2
print(s.scenario_type) # -> traversing_traffic_light_intersection
Why does the same scenario (given its token) have a different type depending on how I define the filters?!
Debugging this led me to the following line where one of multiple tags is arbitrarily selected:
https://github.com/motional/nuplan-devkit/blob/d60b4cd2071de9bb041509c43f5226dd22f248c0/nuplan/database/nuplan_db/nuplan_scenario_queries.py#L888
If a type filter is applied, the returned set of types will only return a single entry in the first place, which is why we end up with two different results:
https://github.com/motional/nuplan-devkit/blob/d60b4cd2071de9bb041509c43f5226dd22f248c0/nuplan/database/nuplan_db/nuplan_scenario_queries.py#L812-L818
This is super confusing in two ways: first, why is the scenario_type property of NuplanScenario a single string in the first place, instead of a list, if a scenario can have multiple types? And, second, the devkit should at least set that type consistently.