Some cases give misleading "on_route" information

[jkewang]

the "on_route" information of the map is wrong in some cases, which will mislead the agent to the wrong way. For example: in scene token "5628b7480cb352db", Our model behave like:

Firstly, I thought it's a bad case for our model. However, when I draw the scene offline, the result shows like this:

As shown in the figure, the thick centerlines represent theirs on_route=1, and the thin centerlines represent theirs on_route=0. Therefore, this figure shows that the route given for this scene is wrong.

Since this scenario is something we stumbled across, it's not clear how many scenarios with the same problem exist. If the ratio is large, it will seriously affect the test results. Please confirm the source and severity of the problem.

[patk-motional]

Hi @jkewang,

Which db file is this from? In the meantime, you can try to use the filter scenario_filter.remove_invalid_goals=true to help with this.

As for the competition, we manually checked all the scenarios used. The routes and goals should all be valid.