TorchSSC

Implement some state-of-the-art methods of Semantic Scene Completion (SSC) task in PyTorch.

Highlights:

• Distributed training
• Easy-to-modify benchmark code
• High performance

News

• 2020/07/28

  Code release for the paper 3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior, CVPR 2020. [arXiv], [Supplementary Material and Demo]

Performance

NYU

Data Preparation && Environment Installation

Pretrained ResNet-50

Please download the pretrained ResNet-50 and then put it into ./DATA/pytorch-weight.

NYU Depth V2

Please download NYU dataset and then put it into ./DATA/NYU.

Environment Installation

Please refer to this documentation

3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior

Training and Inference

Training

Training on NYU Depth V2:

$ cd ./model/sketch.nyu
$ export NGPUS=2
$ python -m torch.distributed.launch --nproc_per_node=$NGPUS train.py -p 10097

• -p is the port number. It is about the distributed training. If you run more than one experiments in the same machine, you should set different ports for them.
• The tensorboard file is saved in sketch.nyu/log/tb/ directory.

Inference

Inference on NYU Depth V2:

$ cd ./model/sketch.nyu
$ python eval.py -e 200-250 -d 0-1 --save_path results

• Here, 200-250 means we evaluate on checkpoints whose ID is in [200, 250], such as epoch-200.pth, epoch-249.pth, etc.
• The SSC predictions will be saved in results/ and results_sketch/, the former stores the SSC predictions and the latter stores sketch preditcions. Performance will be written to log/*.log. You will expect 0.411@SSC mIoU and 0.713@SC IoU.

Citation

If you find this work useful in your research, please consider cite:

@InProceedings{Chen_2020_SketchAwareSSC,
    author = {Chen, Xiaokang and Lin, Kwan-Yee and Qian, Chen and Zeng, Gang and Li, Hongsheng},
    title = {3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    month = {June},
    year = {2020}
}

Acknowledgement

Thanks TorchSeg for their excellent project!

TODO

• [ ] Code on more datasets (NYUCAD/SUNCG).
• [ ] More SSC models.