how to evaluate on custom dataset?

[InfiniteSamele]

I have successfully trained models on my custom dataset, however, when it went to the test phase, it showed me the following error, i thought the reason is custom_dataset.py doesn't have generate_prediction_dicts method. And after I read the CUSTOM_DATASET_TOTURIAL.md, it's just too simple about the Evaluation part, I am still confused about how to evalute my model.

2022-09-17 15:28:15,040 INFO Start evaluation custon_models/pv_rcnn(default) 2022-09-17 15:28:15,041 INFO Loading Custom dataset. 2022-09-17 15:28:15,042 INFO Total samples for CUSTOM dataset: 150 2022-09-17 15:28:15,043 INFO ==> Loading parameters from checkpoint /home/service/work_space/OpenPCDet/output/custon_models/pv_rcnn/default/ckpt/checkpoint_epoch_100.pth to GPU 2022-09-17 15:28:15,106 INFO ==> Checkpoint trained from version: pcdet+0.5.2+dadda9e 2022-09-17 15:28:15,117 INFO ==> Done (loaded 319/319) 2022-09-17 15:28:15,120 INFO *************** EPOCH 100 EVALUATION ***************** eval: 0%| | 0/150 [00:00<?, ?it/s]Traceback (most recent call last): File "/home/service/work_space/OpenPCDet/tools/train.py", line 203, in main() File "/home/service/work_space/OpenPCDet/tools/train.py", line 193, in main repeat_eval_ckpt( File "/home/service/work_space/OpenPCDet/tools/test.py", line 120, in repeat_eval_ckpt tb_dict = eval_utils.eval_one_epoch( File "/home/service/work_space/OpenPCDet/tools/eval_utils/eval_utils.py", line 65, in eval_one_epoch det_annos += annos TypeError: 'NoneType' object is not iterable eval: 0%| | 0/150 [00:00<?, ?it/s]

Process finished with exit code 1

And the demo.py also gives me an error, can you told me how to fix that?

2022-09-17 17:28:32,067 INFO -----------------Quick Demo of OpenPCDet------------------------- 2022-09-17 17:28:32,068 INFO Total number of samples: 1 2022-09-17 17:28:33,740 INFO ==> Loading parameters from checkpoint ../output/custon_models/pv_rcnn/default/ckpt/checkpoint_epoch_100.pth to CPU 2022-09-17 17:28:33,813 INFO ==> Checkpoint trained from version: pcdet+0.5.2+dadda9e 2022-09-17 17:28:33,830 INFO ==> Done (loaded 319/319) 2022-09-17 17:28:34,034 INFO Visualized sample index: 1 Traceback (most recent call last): File "/home/service/work_space/OpenPCDet/tools/demo.py", line 112, in main() File "/home/service/work_space/OpenPCDet/tools/demo.py", line 100, in main V.draw_scenes( File "/home/service/work_space/OpenPCDet/tools/visual_utils/open3d_vis_utils.py", line 70, in draw_scenes vis = draw_box(vis, ref_boxes, (0, 1, 0), ref_labels, ref_scores) File "/home/service/work_space/OpenPCDet/tools/visual_utils/open3d_vis_utils.py", line 109, in draw_box line_set.paint_uniform_color(box_colormap[ref_labels[i]]) IndexError: list index out of range

[jihanyang]

In my toy case, it can evaluate directly if all config are set well, so we don't write any tutorial for evaluation. In you case, could you check why this is None?

File "/home/service/work_space/OpenPCDet/tools/eval_utils/eval_utils.py", line 65, in eval_one_epoch
det_annos += annos
TypeError: 'NoneType' object is not iterable

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.