godot_rl_agents icon indicating copy to clipboard operation
godot_rl_agents copied to clipboard

Refactor and improve the Camera Sensor and example

Open edbeeching opened this issue 2 years ago • 3 comments

edbeeching avatar Mar 01 '23 09:03 edbeeching

This is currently in progress: https://github.com/edbeeching/godot_rl_agents_examples/pull/14#issuecomment-1837311171

This should fix the sensor for Godot 4.2 so it works during training.

What will still be left is to look into:

  • [ ] Onnx export / inference support (I didn't yet test whether this works)
  • [x] Add downsampling option for rendered images before sending as obs
  • [ ] Potential grayscale feature (reduce the size where colors are not necessary)
  • [ ] Maybe having an option to add depth information as the 4th channel

As well as any other features that might be useful.

Ivan-267 avatar Dec 03 '23 10:12 Ivan-267

Hey @Ivan-267 are any items left to do on this?

edbeeching avatar Feb 13 '24 10:02 edbeeching

Those were just some proposed options, none are implemented yet. I've tried the sensor a bit after writing those and I can confirm that we can't export onnx / run inference right now.

Edit: I added the idea about downsampling from discord here as well. The list is sorted by importance, to fully support the sensor we need onnx export, other enhancements are optional.

Ivan-267 avatar Feb 13 '24 11:02 Ivan-267