waymo-open-dataset
waymo-open-dataset copied to clipboard
Sim agent challenge V1 metric testing score
Hi there! I am currently working on a project using Sim agent challenges.
To form a more complete comparison with prior works, I would like to compare my method with 2023 challenge methods (e.g., Trajeglish). However, as the submission for the 2023 challenge is closed and the metric for 2024 challenge changed, I could not get a testing score to compare with the methods of the 2023 challenge.
Is there a chance that I can get V1 metrics for my current testing submission to 2024 challenge?
On the other hand, if that's not possible, how far would the validation score for V1 metrics (which I can compute offline) be from the testing scores? (If they are not too far away, I can have a rough comparison using validation scores for 2023 challenge methods).
Thank you so much for your help in advance!