UVTR
UVTR copied to clipboard
Published
20 hours ago •
dvlab-research
dvlab-research
low results on nusenes-mini
Hi, I was trying to evaluate the pretrained models on the nuscenes mini data, but I am getting very bad results which is unexpected. I have mmdet3d v1.0.0.rc4 installed
Do you maybe know what the problem is?
Thanks :)
here is the log
user@gpu-cluster:~/src/mmdet$ python3 extra_tools/test.py configs/uvtr/multi_modality/uvtr_m_v0075_r101_h5.py models/uvtr/uvtr_m_v0075_r101_h5.pth --eval=bbox --out mini_test.pkl
/home/user/.local/lib/python3.8/site-packages/mmcv/__init__.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
warnings.warn(
/home/user/.local/lib/python3.8/site-packages/mmdet/utils/setup_env.py:38: UserWarning: Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
warnings.warn(
/home/user/.local/lib/python3.8/site-packages/mmdet/utils/setup_env.py:48: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
warnings.warn(
/usr/local/lib/python3.8/dist-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3190.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
load checkpoint from local path: models/uvtr/uvtr_m_v0075_r101_h5.pth
2023-04-25 16:42:17,693 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.0.conv2 is upgraded to version 2.
2023-04-25 16:42:17,695 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.1.conv2 is upgraded to version 2.
2023-04-25 16:42:17,697 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.2.conv2 is upgraded to version 2.
2023-04-25 16:42:17,699 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.3.conv2 is upgraded to version 2.
2023-04-25 16:42:17,701 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.4.conv2 is upgraded to version 2.
2023-04-25 16:42:17,703 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.5.conv2 is upgraded to version 2.
2023-04-25 16:42:17,704 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.6.conv2 is upgraded to version 2.
2023-04-25 16:42:17,706 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.7.conv2 is upgraded to version 2.
2023-04-25 16:42:17,708 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.8.conv2 is upgraded to version 2.
2023-04-25 16:42:17,710 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.9.conv2 is upgraded to version 2.
2023-04-25 16:42:17,712 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.10.conv2 is upgraded to version 2.
2023-04-25 16:42:17,714 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.11.conv2 is upgraded to version 2.
2023-04-25 16:42:17,716 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.12.conv2 is upgraded to version 2.
2023-04-25 16:42:17,717 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.13.conv2 is upgraded to version 2.
2023-04-25 16:42:17,719 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.14.conv2 is upgraded to version 2.
2023-04-25 16:42:17,721 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.15.conv2 is upgraded to version 2.
2023-04-25 16:42:17,723 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.16.conv2 is upgraded to version 2.
2023-04-25 16:42:17,725 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.17.conv2 is upgraded to version 2.
2023-04-25 16:42:17,728 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.18.conv2 is upgraded to version 2.
2023-04-25 16:42:17,730 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.19.conv2 is upgraded to version 2.
2023-04-25 16:42:17,732 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.20.conv2 is upgraded to version 2.
2023-04-25 16:42:17,733 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.21.conv2 is upgraded to version 2.
2023-04-25 16:42:17,735 - root - INFO - ModulatedDeformConvPack img_backbone.layer3.22.conv2 is upgraded to version 2.
2023-04-25 16:42:17,737 - root - INFO - ModulatedDeformConvPack img_backbone.layer4.0.conv2 is upgraded to version 2.
2023-04-25 16:42:17,741 - root - INFO - ModulatedDeformConvPack img_backbone.layer4.1.conv2 is upgraded to version 2.
2023-04-25 16:42:17,744 - root - INFO - ModulatedDeformConvPack img_backbone.layer4.2.conv2 is upgraded to version 2.
[ ] 0/81, elapsed: 0s, ETA:/usr/local/lib/python3.8/dist-packages/torch/nn/functional.py:4227: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
/src/mmdet3d/core/bbox/coders/nms_free_coder.py:67: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
self.post_center_range = torch.tensor(
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 81/81, 2.6 task/s, elapsed: 32s, ETA: 0s
writing results to mini_test.pkl
Formating bboxes of pts_bbox
Start to convert detection format...
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 81/81, 23.1 task/s, elapsed: 3s, ETA: 0s
Results writes to /tmp/tmps9zh56ms/results/pts_bbox/results_nusc.json
Evaluating bboxes of pts_bbox
mAP: 0.1188
mATE: 0.8707
mASE: 0.5113
mAOE: 0.6999
mAVE: 1.0449
mAAE: 0.4827
NDS: 0.2029
Eval time: 4.8s
Per-class results:
Object Class AP ATE ASE AOE AVE AAE
car 0.204 0.872 0.186 0.334 0.212 0.065
truck 0.075 0.673 0.290 0.355 0.216 0.118
bus 0.017 1.250 0.216 0.406 3.521 1.000
trailer 0.000 1.000 1.000 1.000 1.000 1.000
construction_vehicle 0.000 1.000 1.000 1.000 1.000 1.000
pedestrian 0.319 0.582 0.308 0.673 0.572 0.367
motorcycle 0.212 0.869 0.317 0.922 0.070 0.011
bicycle 0.095 0.636 0.372 0.609 1.767 0.301
traffic_cone 0.265 0.825 0.425 nan nan nan
barrier 0.000 1.000 1.000 1.000 nan nan
{'pts_bbox_NuScenes/car_AP_dist_0.5': 0.006, 'pts_bbox_NuScenes/car_AP_dist_1.0': 0.1039, 'pts_bbox_NuScenes/car_AP_dist_2.0': 0.2847, 'pts_bbox_NuScenes/car_AP_dist_4.0': 0.4234, 'pts_bbox_NuScenes/car_trans_err': 0.8717, 'pts_bbox_NuScenes/car_scale_err': 0.1856, 'pts_bbox_NuScenes/car_orient_err': 0.3339, 'pts_bbox_NuScenes/car_vel_err': 0.2123, 'pts_bbox_NuScenes/car_attr_err': 0.0646, 'pts_bbox_NuScenes/mATE': 0.8707, 'pts_bbox_NuScenes/mASE': 0.5113, 'pts_bbox_NuScenes/mAOE': 0.6999, 'pts_bbox_NuScenes/mAVE': 1.0449, 'pts_bbox_NuScenes/mAAE': 0.4827, 'pts_bbox_NuScenes/truck_AP_dist_0.5': 0.0037, 'pts_bbox_NuScenes/truck_AP_dist_1.0': 0.0371, 'pts_bbox_NuScenes/truck_AP_dist_2.0': 0.0762, 'pts_bbox_NuScenes/truck_AP_dist_4.0': 0.1825, 'pts_bbox_NuScenes/truck_trans_err': 0.673, 'pts_bbox_NuScenes/truck_scale_err': 0.2899, 'pts_bbox_NuScenes/truck_orient_err': 0.3548, 'pts_bbox_NuScenes/truck_vel_err': 0.2161, 'pts_bbox_NuScenes/truck_attr_err': 0.118, 'pts_bbox_NuScenes/construction_vehicle_AP_dist_0.5': 0.0, 'pts_bbox_NuScenes/construction_vehicle_AP_dist_1.0': 0.0, 'pts_bbox_NuScenes/construction_vehicle_AP_dist_2.0': 0.0, 'pts_bbox_NuScenes/construction_vehicle_AP_dist_4.0': 0.0, 'pts_bbox_NuScenes/construction_vehicle_trans_err': 1.0, 'pts_bbox_NuScenes/construction_vehicle_scale_err': 1.0, 'pts_bbox_NuScenes/construction_vehicle_orient_err': 1.0, 'pts_bbox_NuScenes/construction_vehicle_vel_err': 1.0, 'pts_bbox_NuScenes/construction_vehicle_attr_err': 1.0, 'pts_bbox_NuScenes/bus_AP_dist_0.5': 0.0, 'pts_bbox_NuScenes/bus_AP_dist_1.0': 0.0, 'pts_bbox_NuScenes/bus_AP_dist_2.0': 0.0003, 'pts_bbox_NuScenes/bus_AP_dist_4.0': 0.0676, 'pts_bbox_NuScenes/bus_trans_err': 1.2499, 'pts_bbox_NuScenes/bus_scale_err': 0.216, 'pts_bbox_NuScenes/bus_orient_err': 0.4058, 'pts_bbox_NuScenes/bus_vel_err': 3.5214, 'pts_bbox_NuScenes/bus_attr_err': 1.0, 'pts_bbox_NuScenes/trailer_AP_dist_0.5': 0.0, 'pts_bbox_NuScenes/trailer_AP_dist_1.0': 0.0, 'pts_bbox_NuScenes/trailer_AP_dist_2.0': 0.0, 'pts_bbox_NuScenes/trailer_AP_dist_4.0': 0.0, 'pts_bbox_NuScenes/trailer_trans_err': 1.0, 'pts_bbox_NuScenes/trailer_scale_err': 1.0, 'pts_bbox_NuScenes/trailer_orient_err': 1.0, 'pts_bbox_NuScenes/trailer_vel_err': 1.0, 'pts_bbox_NuScenes/trailer_attr_err': 1.0, 'pts_bbox_NuScenes/barrier_AP_dist_0.5': 0.0, 'pts_bbox_NuScenes/barrier_AP_dist_1.0': 0.0, 'pts_bbox_NuScenes/barrier_AP_dist_2.0': 0.0, 'pts_bbox_NuScenes/barrier_AP_dist_4.0': 0.0, 'pts_bbox_NuScenes/barrier_trans_err': 1.0, 'pts_bbox_NuScenes/barrier_scale_err': 1.0, 'pts_bbox_NuScenes/barrier_orient_err': 1.0, 'pts_bbox_NuScenes/barrier_vel_err': nan, 'pts_bbox_NuScenes/barrier_attr_err': nan, 'pts_bbox_NuScenes/motorcycle_AP_dist_0.5': 0.0019, 'pts_bbox_NuScenes/motorcycle_AP_dist_1.0': 0.1057, 'pts_bbox_NuScenes/motorcycle_AP_dist_2.0': 0.2772, 'pts_bbox_NuScenes/motorcycle_AP_dist_4.0': 0.4626, 'pts_bbox_NuScenes/motorcycle_trans_err': 0.8689, 'pts_bbox_NuScenes/motorcycle_scale_err': 0.3167, 'pts_bbox_NuScenes/motorcycle_orient_err': 0.9223, 'pts_bbox_NuScenes/motorcycle_vel_err': 0.0699, 'pts_bbox_NuScenes/motorcycle_attr_err': 0.0111, 'pts_bbox_NuScenes/bicycle_AP_dist_0.5': 0.0086, 'pts_bbox_NuScenes/bicycle_AP_dist_1.0': 0.0796, 'pts_bbox_NuScenes/bicycle_AP_dist_2.0': 0.1092, 'pts_bbox_NuScenes/bicycle_AP_dist_4.0': 0.1837, 'pts_bbox_NuScenes/bicycle_trans_err': 0.636, 'pts_bbox_NuScenes/bicycle_scale_err': 0.3723, 'pts_bbox_NuScenes/bicycle_orient_err': 0.6093, 'pts_bbox_NuScenes/bicycle_vel_err': 1.7671, 'pts_bbox_NuScenes/bicycle_attr_err': 0.3007, 'pts_bbox_NuScenes/pedestrian_AP_dist_0.5': 0.1008, 'pts_bbox_NuScenes/pedestrian_AP_dist_1.0': 0.2702, 'pts_bbox_NuScenes/pedestrian_AP_dist_2.0': 0.3986, 'pts_bbox_NuScenes/pedestrian_AP_dist_4.0': 0.5082, 'pts_bbox_NuScenes/pedestrian_trans_err': 0.5822, 'pts_bbox_NuScenes/pedestrian_scale_err': 0.3078, 'pts_bbox_NuScenes/pedestrian_orient_err': 0.6728, 'pts_bbox_NuScenes/pedestrian_vel_err': 0.572, 'pts_bbox_NuScenes/pedestrian_attr_err': 0.3673, 'pts_bbox_NuScenes/traffic_cone_AP_dist_0.5': 0.0334, 'pts_bbox_NuScenes/traffic_cone_AP_dist_1.0': 0.2129, 'pts_bbox_NuScenes/traffic_cone_AP_dist_2.0': 0.3517, 'pts_bbox_NuScenes/traffic_cone_AP_dist_4.0': 0.4602, 'pts_bbox_NuScenes/traffic_cone_trans_err': 0.8253, 'pts_bbox_NuScenes/traffic_cone_scale_err': 0.4245, 'pts_bbox_NuScenes/traffic_cone_orient_err': nan, 'pts_bbox_NuScenes/traffic_cone_vel_err': nan, 'pts_bbox_NuScenes/traffic_cone_attr_err': nan, 'pts_bbox_NuScenes/NDS': 0.20291704194649415, 'pts_bbox_NuScenes/mAP': 0.1187504058313846}
