gz-sensors icon indicating copy to clipboard operation
gz-sensors copied to clipboard

Improve support for triggered cameras

Open iche033 opened this issue 10 months ago • 1 comments

Currently a CameraSensor can be configured to be a triggered camera by setting the SDF element to true. A triggered camera does not publish images unless a message is received on the trigger topic.

Here are some proposals for improving support for triggered cameras:

  • Allow instant on-demand camera image rendering:
    • The camera sensor currently still respects it's update_rate, meaning the camera does not update immediately when the trigger msg is received, instead it still waits for its time to update. For simulation running at low RTF, the wait time can be long.
  • Extend support for <triggered> SDF param to other camera sensors
    • currently only supported by CameraSensor. Consider making it generic and extending to other types of camera sensors, e.g. depth, rgbd, etc

Nice to haves:

  • Provide an API to toggle the triggered property of the Camera sensor

Alternatives considered

As opposed to changing the timing on how triggered cameras work, we can could add a new SDF parameter for instant on-demand camera image generation. This preserves existing behavior of triggered cameras. However, I'm now sure if there is a common use case for cameras to respect their update rates when they are triggered.

Implementation suggestion

  • Refactor CameraSensor's logic for loading and handling the SDF param, make it generic so that it can be used by other camera sensor types.
  • In Sensor::Update function, there is a check to see if it's time for a sensor to update. Some options to make a sensor update immediate:
    • set the update_rate to 0
    • set nextUpdateTime to 0
    • add a new bool flag to indicate the sensor has been triggered

iche033 avatar Apr 19 '24 18:04 iche033

<triggered> support for depth cameras would be great, especially for applications where a very high density pointcloud is needed. Currently, that leads to poor RTF and UX on those simulations, since moving around in Gazebo can become very laggy.

mcres avatar Apr 24 '24 09:04 mcres