Evaluation on Mini dataset gives ""Samples in split doesn't match samples in predictions.""

[chowkamlee81]

For training and val dataset , evaluation toolkit works but when i start evaluating on v1.0 mini dataset gives

""Samples in split doesn't match samples in predictions.""

Kindly help

[tjucwb]

Hi, dude, have you solved this problem, I also encountered this problem, THANKS

[tjucwb]

I solved that, just modify line 308 in nuscenes_dataset.py eval_set_map, from "v1.0-mini": "mini_train" to "v1.0-mini": "mini_val", additionally, the size of either set, train_scenes and val_scenes can not be zero

[3846chs]

See tools/dist_test.py#L201.

This problem occurs because v1.0-trainval dataset is loaded. (not v1.0-mini)

Change

result_dict, _ = dataset.evaluation(copy.deepcopy(predictions), output_dir=args.work_dir, testset=args.testset)

to

dataset.version = "v1.0-mini"
result_dict, _ = dataset.evaluation(copy.deepcopy(predictions), output_dir=args.work_dir, testset=args.testset)

This command causes dataset.version to change from v1.0-trainval to v1.0-mini.

[wongyibin]

I used "v1.0-mini“ and meet this problem. I modify this line : eval_set_map = { 'v1.0-mini': 'mini_val', 'v1.0-trainval': 'val', } to : eval_set_map = { 'v1.0-mini': 'mini_train', 'v1.0-trainval': 'val', }

[Byte247]

hmm still happens to me even though it already says: "v1.0-mini": "mini_val"