OpenPCDet icon indicating copy to clipboard operation
OpenPCDet copied to clipboard

Published 20 hours ago • verified publisher icon open-mmlab

APH is all equal to 0

Open SiHengHeHSH opened this issue 1 year ago • 3 comments

why the recall is not 0, but APH and AP is all equal to 0. I expend the Voxel-RCNN based from center head to dense head in waymo

2024-01-04 10:03:12,810 INFO Generate label finished(sec_per_example: 0.5805 second). 2024-01-04 10:03:12,810 INFO recall_roi_0.3: 0.651651 2024-01-04 10:03:12,811 INFO recall_rcnn_0.3: 0.653542 2024-01-04 10:03:12,811 INFO recall_roi_0.5: 0.531293 2024-01-04 10:03:12,811 INFO recall_rcnn_0.5: 0.554161 2024-01-04 10:03:12,811 INFO recall_roi_0.7: 0.326857 2024-01-04 10:03:12,811 INFO recall_rcnn_0.7: 0.402682 2024-01-04 10:03:12,811 INFO Average predicted number of objects(199 samples): 63.010 2024-01-04 10:03:12.860708: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2024-01-04 10:03:13.210926: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0. 2024-01-04 10:03:13.974207: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/hsh/anaconda3/envs/waymo_voxel/lib/python3.9/site-packages/cv2/../../lib64: 2024-01-04 10:03:13.974278: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/hsh/anaconda3/envs/waymo_voxel/lib/python3.9/site-packages/cv2/../../lib64: 2024-01-04 10:03:13.974283: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. Start the waymo evaluation... Number: (pd, 12539) VS. (gt, 0) Level 1: 0, Level2: 0) WARNING:tensorflow:From /home/hsh/anaconda3/envs/waymo_voxel/lib/python3.9/contextlib.py:87: TensorFlowTestCase.test_session (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version. Instructions for updating: Use self.session() or self.cached_session() instead. 2024-01-04 10:03:14.771745: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2024-01-04 10:03:14.772075: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/hsh/anaconda3/envs/waymo_voxel/lib/python3.9/site-packages/cv2/../../lib64: 2024-01-04 10:03:14.772085: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform. Skipping registering GPU devices... 2024-01-04 10:03:14.772694: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2024-01-04 10:03:14.778603: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:357] MLIR V1 optimization pass is not enabled 2024-01-04 10:03:14.852481: W tensorflow/c/c_api.cc:291] Operation '{name:'Const_22' id:340 op device:{requested: '', assigned: ''} def:{{{node Const_22}} = Const_has_manual_control_dependencies=true, dtype=DT_FLOAT, value=Tensor<type: float shape: [0] values: >}}' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session. 2024-01-04 10:03:14.903115: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:14.903146: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:14.906163: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:14.948107: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:14.958517: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:14.958543: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:14.961580: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:14.984499: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:14.993187: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:14.993229: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:14.996111: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.015813: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.025085: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.025101: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.026989: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.042019: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.051641: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.051672: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.054364: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.066115: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.075412: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.075445: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.078083: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.087614: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.097619: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.097643: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.099811: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.122872: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.133583: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.133610: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.135618: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.153519: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.163238: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.163272: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.166040: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.183367: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.193207: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.193244: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.196142: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.208428: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.218031: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.218065: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.220905: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.234148: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.244452: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.244474: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.246661: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.258670: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.268220: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.268245: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.270318: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.279664: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.289749: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.289781: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.292419: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.312593: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.322932: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.322954: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.325071: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.336869: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.346815: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.346838: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.348756: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.360028: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.369501: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.369523: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.371470: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.383592: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.394821: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.394865: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.398095: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.408502: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.419448: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.419471: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.421263: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.431223: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.440319: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.440354: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.442155: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.455662: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.466778: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.466813: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.469526: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.481982: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.492337: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.492360: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.494425: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.506528: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.515641: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.515675: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.517637: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.529312: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15.539287: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:188] Computing detection metrics for 12539 predicted boxes. 2024-01-04 10:03:15.539313: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:190] Parsing prediction [12539,7][12539] 2024-01-04 10:03:15.541373: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:199] Parsing ground truth [0,7][0] 2024-01-04 10:03:15.558801: I waymo_open_dataset/metrics/ops/detection_metrics_ops.cc:252] Done with computing detection metrics. 2024-01-04 10:03:15,564 INFO
OBJECT_TYPE_TYPE_VEHICLE_LEVEL_1/AP: 0.0000 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_1/APH: 0.0000 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_1/APL: 0.0000 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_2/AP: 0.0000 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_2/APH: 0.0000 OBJECT_TYPE_TYPE_VEHICLE_LEVEL_2/APL: 0.0000 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1/AP: 0.0000 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1/APH: 0.0000 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1/APL: 0.0000 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2/AP: 0.0000 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2/APH: 0.0000 OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2/APL: 0.0000 OBJECT_TYPE_TYPE_SIGN_LEVEL_1/AP: 0.0000 OBJECT_TYPE_TYPE_SIGN_LEVEL_1/APH: 0.0000 OBJECT_TYPE_TYPE_SIGN_LEVEL_1/APL: 0.0000 OBJECT_TYPE_TYPE_SIGN_LEVEL_2/AP: 0.0000 OBJECT_TYPE_TYPE_SIGN_LEVEL_2/APH: 0.0000 OBJECT_TYPE_TYPE_SIGN_LEVEL_2/APL: 0.0000 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_1/AP: 0.0000 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_1/APH: 0.0000 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_1/APL: 0.0000 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_2/AP: 0.0000 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_2/APH: 0.0000 OBJECT_TYPE_TYPE_CYCLIST_LEVEL_2/APL: 0.0000

SiHengHeHSH avatar Jan 04 '24 02:01 SiHengHeHSH

Hi!

I've encountered an issue regarding evaluation duration. When training my model, the time spent on "Generate label finished" significantly exceeds the time spent on training or evaluation, reaching an unbearable duration.

I noticed that the time you spent on generating results in your logs is quite short. Could you please provide me with some information, such as the versions of various libraries in your virtual environment (conda env) , the version of the Waymo dataset you're using, and the processing method (multi-frame or single-frame), or any other potentially useful details?

Thanks!

Rzlnmc avatar Jan 15 '24 13:01 Rzlnmc

I use the waymo v1.0, so the APH is all zero.Thank you!

SiHengHeHSH avatar Jan 18 '24 02:01 SiHengHeHSH

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] avatar Feb 18 '24 01:02 github-actions[bot]

I met the same issue, can you tell me how to solve it ? thanks!

fx110127 avatar Feb 26 '24 03:02 fx110127

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] avatar Mar 28 '24 01:03 github-actions[bot]

This issue was closed because it has been inactive for 14 days since being marked as stale.

github-actions[bot] avatar Apr 11 '24 01:04 github-actions[bot]