depthai icon indicating copy to clipboard operation
depthai copied to clipboard

Ability to return location in pixel space or xyz of generic depth blobs above certain size

Open Luxonis-Brandon opened this issue 5 years ago • 0 comments

Start with the why:

In many applications that involve obstacle avoidance, it doesn’t matter what the obstacle is, i.e. the obstacles encountered are not conveniently in a PASCAL-VOC-2007 20-class set, and in fact, the type of obstacles to be avoided could effectively be semi-infinite. So using a trained neural network for such obstacle detection may not be applicable in this case (as the training data may not be sufficient to detect a semi-infinite number of obstacles).

So enable use of DepthAI to solve such obstacle avoidance problems, one technique is to be able to mark an obstacle which is above a certain threshold above a plane.

This is then applicable to many common robotic navigation cases where for the most part the robot is/should-be operating on a plane (sidewalk, road, warehouse aisle, grass field, etc.) and it is desirable for the robot to remain in the planar areas and detect objects (and/or voids, i.e. holes) which deviate too much from this plane.

Move to the how:

This feature is likely best implemented leveraging the coming IMU support addition to DepthAI (https://github.com/luxonis/depthai-hardware/issues/8).

So in this way, the IMU can assist in plane estimation in conjunction with computer vision. Upon implementation, a literature review for plane detection and obstacle-above-a-plane detection is likely in order.

Move to the what:

  1. Plane detection
  2. Capability to mark something above a volume-threshold/height-threshold above plane as an obstacle.
  3. Capability for DepthAI API to set the volume and/or height-threshold.

Luxonis-Brandon avatar Jul 07 '20 19:07 Luxonis-Brandon