zed-ros2-wrapper icon indicating copy to clipboard operation
zed-ros2-wrapper copied to clipboard

Published 20 hours ago • verified publisher icon stereolabs

CUSTOM_BOX_OBJECTS support

Open osu-uwrt-bot opened this issue 2 years ago • 8 comments

Preliminary Checks

  • [X] This issue is not a duplicate. Before opening a new issue, please search existing issues.
  • [X] This issue is not a question, bug report, or anything other than a feature request directly related to this project.

Proposal

Support for the CUSTOM_BOX_OBJECTS from the ZED object detection module. The ZED SDK has a parameter which allows users to submit custom detections to it and it is able to track the objects as well as output the position and 3d bounding boxes for the objects.

Currently the wrapper doesn't expose that part of the SDK and it isn't possible to access that part of the SDK through another ROS node while running the wrapper due to the wrapper having control of the camera.

Would it be possible to enable that as an option within the launch file and then accept the objects as a message to which the wrapper outputs a Detections3Darray (or even a custom message with the info from the SDK).

Use-Case

I would be able to use a custom detector in ROS with 3D bounding boxes

Anything else?

No response

osu-uwrt-bot avatar Apr 26 '22 19:04 osu-uwrt-bot

We are working on this feature. The only issue is the latency:

  1. The zed_node publishes the rgb image (dT_1)
  2. The detector_node receives the rgb image (dT_2)
  3. The detector_node extracts the 2D bounding box and publishes it (dT_3)
  4. The zed_node receives the 2D bbox and starts the tracking

If dT_1+dT_2+dT_3<T_grab then there is only one frame of latency and the tracking can work correctly, otherwise it can be not so precise.

Furthermore, ROS introduces latency in the communication process... to know if this engine works we can only put all the gears together and test it.

Myzhar avatar Apr 26 '22 19:04 Myzhar

Hi @Myzhar, how is the progress of this feature? Will it be implemented anytime soon?

MrOCW avatar Oct 07 '22 02:10 MrOCW

Hi @Myzhar any progress on that? Would it be possible to change the underlying model of the base functions and theoretically have our own custom detection model run?

Marketos-Damigos avatar Jan 30 '23 18:01 Marketos-Damigos

I'm sorry but this is not currently planned. We suggest you add custom detection to the main code.

Myzhar avatar Jan 30 '23 18:01 Myzhar

I'm sorry but this is not currently planned. We suggest you add custom detection to the main code.

Could you elaborate on how to approach it?

Marketos-Damigos avatar Jan 30 '23 20:01 Marketos-Damigos

He commented here https://community.stereolabs.com/t/custom-object-detection-with-ros1/1527

shaoyifei96 avatar Feb 08 '23 02:02 shaoyifei96

Any news about this resource? It would be very helpful in my project.

Meanwhile, I'll be working on a way to do this following the approach suggested above.

Mat198 avatar Sep 28 '23 17:09 Mat198

Hi, did anyone actually implemented a custom detector in ROS2?

creminem94 avatar Apr 23 '24 13:04 creminem94