mask for detector

[pak-da]

Hi, I am using find_object_2d for a single object detection and manipulation task with a robot arm.The object is well known so I can detect it even with a single RGB camera. I need precious and robust result that seems to be reached with SIFT detector and description however I'm sure that I can do something better with a color based mask in order to filter out all the keypoints that can be similar but belong to a region with different color. So how can I impose a mask for detector?

Thanks

[matlabbe]

The function here: https://github.com/introlab/find-object/blob/fd9270a5617f31087bc61c3fae919961a2240b78/src/MainWindow.cpp#L1318 can use a mask as parameter. You may create a mask from the input depth image topic like this. You would have to create a fake depth image from the color mask, then feed that depth image to find_object_node.

[pak-da]

It seems that findObject::detect doesn't accept directly a mask. https://github.com/introlab/find-object/blob/fd9270a5617f31087bc61c3fae919961a2240b78/src/FindObject.cpp#L1423

However it uses the class ExtractFeaturesThread https://github.com/introlab/find-object/blob/fd9270a5617f31087bc61c3fae919961a2240b78/src/FindObject.cpp#L1447

in which the mask seems just initialized https://github.com/introlab/find-object/blob/fd9270a5617f31087bc61c3fae919961a2240b78/src/FindObject.cpp#L749-L756

If I've understood fine you are suggesting to replace the initialization of the mask with a variable mask that is the 4th channel. Of course I have to give as input a 4 channel image. Is it right?

I'm not very experienced in programming in C++ so if you could give me some examples i would be grateful

[matlabbe]

If you are using ROS, we can make rtabmap subscribe to a depth image (single channel 16bits unsigned short, or CV_16UC1). The depth image is passed here before FindObject::detect() gets called: https://github.com/introlab/find-object/blob/fd9270a5617f31087bc61c3fae919961a2240b78/src/MainWindow.cpp#L1296

As I said, you can add a new parameter to the function FindObject::detect(..., const cv::Mat & mask), and you can convert the depth to a mask matrix (CV_8UC1) using the conversion code I linked to pass the mask to that function. The mask can be then redirected to ExtractFeaturesThread.

Regards, Mathieu

[pak-da]

I've tried what you suggested and it works fine. Thanks Is there any way in order to avoid the tf publishing? (it is wrong because the depth channel is fake)

[matlabbe]

You could set if(0) here: https://github.com/introlab/find-object/blob/fd9270a5617f31087bc61c3fae919961a2240b78/src/ros/FindObjectROS.cpp#L61