USIP
USIP copied to clipboard
lijx10
Code Release.
Hi, I really appreciate your work on point cloud keypoint detection. Is there any estimated date for the release? Thank you very much.
Yes, definitely the code will be released in the near future. We are still waiting for our paper acceptance. We wish to release the code as soon as possible as well.
Do you have an estimated date for the release ? And do you plan to work on a version 2?
The codes will be released tomorrow. Thanks for your patience.
@tchaton Yes, it is in pytorch. I don't have ideas for version 2 so far.
@lijx10, Awesome :) I will give it a try then. For a version 2 (not worth a paper) would be to replace the backbone of your model (pointNet) by something state-of-the-art (like ShellNet or DGCNN) and outperforms previous method while being even faster.
@lijx10, and it would be great to know if this model is able to perform good registration over indoor point cloud like s3dis or scannet.
And how precise it is in term of meters, centimeters, mm for real case applications
@tchaton The codes are online now. Yes, replacing the backbone is an option, but we are not prioritizing it since our major contribution is the idea of unsupersied keypoint detection. Please feel free the give it a try and share the results with us. Thanks!
Our method works for general point clouds, including indoor scans. However, the results for single frame RGBD (sceneNN, redwood, 3dmatch) is less appealing (but we still out-perform state-of-the-art), compared to the superior results for outdoor / object model. But we haven't tried with large scale indoor scenes yet.
@lijx10 Thanks you very much for both releasing the code and your answer. I think this could be interesting for me, but from your paper, it seems that the point cloud are overlapping .
In our case, we are building indoors scans and we try to do registration using AI at scale. Potentially having hundreds of sub point cloud where sometimes the overlap is almost none.
Here could be an idea for a version 2: If possible, your model could have an attention mechanisms in 3D space allowing him to look for keypoints pairs in sub part of the space.
But doing so, it will allow the model to perform keypoints maching even if the overlap between point cloud is small.
It is highly recommended to have at least 60 % overlap, but in your case (after asking scanning teams), it is closer to 30 % in average.
Sorry but I am confused, how can we perform registration if there is no overlap?
@lijx10 In real case, the overlap is usually between 30% to 60 %. The open source datasets are usually extremely far from real world data, and those are no exceptions. Therefore, the real question is how can we perform accurate registration with such a small coverage. Currently, people provide at least a minimum of 3 keypoint pairs which my help to define the search zone used for registration. They give an estimated transformation matrix and the idea is to refine in using the overlap description. Ideally, it will be great to have a model able to find this potential zone of overlap (assuming at least 30 % overlap) and perform registration by giving keypoint pairs. Your research would become extremely valuable for the entire community if it was providing a study of how it performs with small overlapped point cloud + (potential improvement for this real world use case).
Best, Thomas Chaton
Hi @lijx10 Thanks for your sharing. Are you also going to share the evaluation code?
@tchaton Currently the USIP paper is only about feature / keypoint detection and description. Point cloud registration is performed with standard RANSAC pipeline. In practice, the RANSAC pipeline is able to handle small overlaps, as long as the feature descriptor is good enough.
We have experiments with Redwood dataset registration, where the overlap is as low as 30%. Our registration recall is 76.4%, which out-performs state-of-the-art as well. However, it is not a huge improvement. As shown in the USIP paper, we didn't achieve very good results on RGBD scans. For my experiments, RGBD scans are small scale, and contain lots of flat surface like walls, floors, etc., therefore, feature descriptor matters more than feature detector.