PCR-CG: PCR-CG: Point Cloud Registration via Color and Geometry. [ECCV 2022]

Yu Zhang, Junle Yu, Xiaolin Huang, Wenhui Zhou, Ji Hou

[Paper] [Video]

This repository represents the official implementation

Introduction

Install

The code is build upon predator. Follow the predator to install the corresponding Python library:

Data:3DLoMatch and 3DMatch(Indoor)

Download our preprocessed data

BaiduNetDisk

Password：axvz

RGB-D DATA: unzip all the 3dmatch-image.rar files into one directory and change the target image path. The full RGB-D DATA occupies 130+ GB, thus we pick 5 frames from 50 frames evenly and upload this file.

SuperGlue preprocessed data: unzip the dump-match-pairs-160.zip file.

RGBD DATA

Check the offical 3DMatch webiste to download the full rgb-d data.Our preprocess data will be uploaded in few days. Follow the Pri3d to download the well pretrained 2d backbone. Or use the the image-net pretrained model direcctly,you can get a near result.

Generating explicit 2d correspondences and Dump image matches :

Superglue offical repository

In our paper,we use two imgs and we choose the first and last image of 50 images(Each 3DMatch point cloud is fused by 50 frames of images).

Generate the 2d matches npz for train/val/test dataset.

Prepocee all the image pairs in text file and dump the keypoints and matches to compressed numpy npz files. Follow Superglue to dump the images pairs using 'match_pairs.py'

python  match_pairs.py

Train

After creating the virtual environment and downloading the corresponding data, PCR-CG can be trained using:

python main.py configs/train/indoor.yaml

modify the img_path and dataset path in configs/train/indoor.yaml as yours.

Evaluate

python main.py configs/test/indoor.yaml

Then using RANSAC to get the estimated transformation: modify the source_path and res_path

sh run_ransac.sh

Citation

If you find our work helpful, please consider citing

@inproceedings{zhang2022pcr,
  title={PCR-CG: Point Cloud Registration via Deep Explicit Color and Geometry},
  author={Zhang, Yu and Yu, Junle and Huang, Xiaolin and Zhou, Wenhui and Hou, Ji},
  booktitle={European Conference on Computer Vision},
  pages={443--459},
  year={2022},
  organization={Springer}
}

Acknowledgments

In this project we use (parts of) the official implementations of the followin works:

• predator (3D backbone)
• CoFiNet (3D backbone)
• GeoTransformer (3D backbone)
• SuperGlue (Extract 2d correspondences)
• Pri3d (Projection Module and 2D backbone)

We thank the respective authors for open sourcing their methods.