I have NuScenes data files on ceph, so I config as https://mmdetection3d.readthedocs.io/en/latest/tutorials/backends_support.html
It works when model training, but when evaluating, it doesn't work.
Evaluating bboxes of pts_bbox
Traceback (most recent call last):
File "tools/train.py", line 257, in
main()
File "tools/train.py", line 253, in main
meta=meta)
File "/mnt/lustre/{username}/workspace/mmdetection3d/mmdet3d/apis/train.py", line 35, in train_model
meta=meta)
File "/mnt/lustre/{username}/workspace/mmdetection/mmdet/apis/train.py", line 244, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/mnt/lustre/{username}/workspace/mmcv/mmcv/runner/epoch_based_runner.py", line 127, in run
epoch_runner(data_loaders[i], **kwargs)
File "/mnt/lustre/{username}/workspace/mmcv/mmcv/runner/epoch_based_runner.py", line 54, in train
self.call_hook('after_train_epoch')
File "/mnt/lustre/{username}/workspace/mmcv/mmcv/runner/base_runner.py", line 307, in call_hook
getattr(hook, fn_name)(self)
File "/mnt/lustre/{username}/workspace/mmcv/mmcv/runner/hooks/evaluation.py", line 267, in after_train_epoch
self._do_evaluate(runner)
File "/mnt/lustre/{username}/workspace/mmdetection/mmdet/core/evaluation/eval_hooks.py", line 125, in _do_evaluate
key_score = self.evaluate(runner, results)
File "/mnt/lustre/{username}/workspace/mmcv/mmcv/runner/hooks/evaluation.py", line 362, in evaluate
results, logger=runner.logger, **self.eval_kwargs)
File "/mnt/lustre/{username}/workspace/mmdetection3d/mmdet3d/datasets/nuscenes_dataset.py", line 505, in evaluate
ret_dict = self._evaluate_single(result_files[name])
File "/mnt/lustre/{username}/workspace/mmdetection3d/mmdet3d/datasets/nuscenes_dataset.py", line 391, in _evaluate_single
version=self.version, dataroot=self.data_root, verbose=False)
File "/mnt/lustre/share_data/{username}/bev/lib/python3.6/site-packages/nuscenes/nuscenes.py", line 62, in init
assert osp.exists(self.table_root), 'Database version not found: {}'.format(self.table_root)
AssertionError: Database version not found: s3://nuscenes/v1.0-trainval
For now, since the nuscenes evaluation relies on the official api, which does not support ceph. If you want to do nuscenes evaluatoin, you need to prepare maps and v1.0-trainval folder under your local data/nuscenes directory.
We'll consider to support ceph for nuscenes evaluation in th future.