openpilot icon indicating copy to clipboard operation
openpilot copied to clipboard

tools/sim: Implement `MetaDrive` simulator

Open jon-chuang opened this issue 2 years ago • 0 comments
trafficstars

Description Add MetaDrive simulator.

Runs at 30 FPS on 1928 X 1208 inputs on an iGPU (Ryzen 6850H - Laptop).

  • Faster (30 FPS), just the openpilot screen: https://www.youtube.com/watch?v=paVD-laxbcY
  • Slower (14 FPS), with the render screen: https://www.youtube.com/watch?v=amLu5dloVaI

We disable render screen.

Misc notes:

  • Refactored bridge.py into clean SimulatorBridge abstraction that shares common logic between CarlaBridge and MetaDriveBridge
  • Monkey patches MetaDrive methods for improved performance
  • We have to force WideCameraOnly for better performance. I particular, we have to run in ExperimentalMode.

Verification Adding an integration test similar to carla simulator's

First part of: https://github.com/commaai/openpilot/issues/27284

Next steps:

  1. Implement streaming img/obs through Docker? (if openpilot UI already spawns from inside Docker, we don't need to)
  2. More e2e tests (https://github.com/commaai/openpilot/issues/26215)

jon-chuang avatar Feb 17 '23 11:02 jon-chuang