coiltraine icon indicating copy to clipboard operation
coiltraine copied to clipboard

Published 20 hours ago • verified publisher icon felipecode

Strongest checkpoint from Nocrash

Open bsande6 opened this issue 2 years ago • 1 comments

I am using the nocrash resnetimnetS1 model as a baseline and have run tests using checkpoint 660000. I am finding that my results for the New Town experiments are much lower than as recorded in the paper Exploring the Limitations of Autonomous Driving. Am I using the same checkpoint as was used in the paper? Are there any reasons that my results on the same experiment would be lower?

bsande6 avatar Apr 29 '22 11:04 bsande6

There are several reasons. Listing some:

  • You are using a very different CARLA version. CARLA changes considerably the pedestrians and car models and that can affect the results. The results were obtained on the old CARLA 0.8.X version.
  • The models used are listed and explained here https://github.com/felipecode/coiltraine/blob/master/docs/exploring_limitations.md. Make sure to use the correct one for reproducing.
  • Make sure you match also the frame rate and configs for the server configuration.

felipecode avatar Apr 29 '22 12:04 felipecode