PillarNeSt: Embracing Backbone Scaling and Pretraining for Pillar-based 3D Object Detection

PillarNeSt is a robust pillar-based 3D object detectors, which obtains 66.9%(SoTA without TTA/model ensemble) mAP and 71.6 % NDS on nuScenes benchmark.

Preparation

• Environments

Python == 3.6
CUDA == 11.1
pytorch == 1.9.0
mmcls == 0.22.1
mmcv-full == 1.4.2
mmdet == 2.20.0
mmsegmentation == 0.20.2
mmdet3d == 0.18.1

• Data
  Follow the mmdet3d to process the nuScenes dataset.

• Weights

Model weights are available at Google Drive and BaiduWangpan(1111).

Main Results

Results on nuScenes val set. (15e + 5e means the last 5 epochs should be trained without GTsample)

Results on nuScenes test set (without any TTA/model ensemble).

Citation

If you find PillarNeSt helpful in your research, please consider citing:

@article{mao2023pillarnest,
  title={PillarNeSt: Embracing Backbone Scaling and Pretraining for Pillar-based 3D Object Detection},
  author={Mao, Weixin and Wang, Tiancai and Zhang, Diankun and Yan, Junjie and Yoshie, Osamu},
  journal={arXiv preprint arXiv:2311.17770},
  year={2023}
}

TODO:

• [ ] weights on test set
• [ ] Backbone code
• [ ] add CenterPoint-Plus
• [x] Small, Base, Large configs
• [x] Upload weights to Baidu cloud

Contact

If you have any questions, feel free to open an issue or contact us at [email protected] ([email protected]) or [email protected].

PS:

Recently, our team also conduct some explorations into the application of multi-modal large language model (MLLM) in the field of autonomous driving:

Adriver-I: A general world model for autonomous driving

@article{jia2023adriver,
  title={Adriver-i: A general world model for autonomous driving},
  author={Jia, Fan and Mao, Weixin and Liu, Yingfei and Zhao, Yucheng and Wen, Yuqing and Zhang, Chi and Zhang, Xiangyu and Wang, Tiancai},
  journal={arXiv preprint arXiv:2311.13549},
  year={2023}
}

PPS:

组内招收自驾大模型、世界模型和具身智能相关的实习生，详情咨询/简历投递：[email protected]