mmrotate
mmrotate copied to clipboard
open-mmlab
The test cannot calculate ap on its own dataset
Prerequisite
- [X] I have searched Issues and Discussions but cannot get the expected help.
- [X] I have read the FAQ documentation but cannot get the expected help.
- [X] The bug has not been fixed in the latest version (master) or latest version (1.x).
Task
I'm using the official example scripts/configs for the officially supported tasks/models/datasets.
Branch
master branch https://github.com/open-mmlab/mmrotate
Environment
sys.platform: win32 Python: 3.8.13 (default, Mar 28 2022, 06:59:08) [MSC v.1916 64 bit (AMD64)] CUDA available: True GPU 0: NVIDIA GeForce GTX 1050 Ti CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2 NVCC: Cuda compilation tools, release 10.2, V10.2.8 MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.29.30146 版 GCC: n/a PyTorch: 1.8.0 PyTorch compiling details: PyTorch built with:
- C++ Version: 199711
- MSVC 192829337
- Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191125 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
- OpenMP 2019
- CPU capability usage: AVX2
- CUDA Runtime 10.2
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
- CuDNN 7.6.5
- Magma 2.5.4
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -DNDEBUG -DUSE_FBGEMM -DUSE_XNNPACK, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON,
TorchVision: 0.9.0 OpenCV: 4.6.0 MMCV: 1.6.0 MMCV Compiler: MSVC 192930137 MMCV CUDA Compiler: 10.2 MMRotate: 0.3.2+unknown
Reproduces the problem - code sample
(mmrotate) PS F:\postgraduatework\mmrotate-main\mmrotate-main> python tools/test.py --config run_model/roi_trans_1/roi_trans_r50_fpn_1x_dota_le90.py --checkpoint run_model/roi_trans_1/latest.pth --eval mAP --show-dir work_dirs/roi_trans_1/result
F:\SoftWare\Anaconda\envs\mmrotate\lib\site-packages\mmdet\models\dense_heads\anchor_head.py:116: UserWarning: DeprecationWarning: num_anchors is deprecated, for consistency or also use num_base_priors instead
warnings.warn('DeprecationWarning: num_anchors is deprecated, '
load checkpoint from local path: run_model/roi_trans_1/latest.pth
[ ] 0/30, elapsed: 0s, ETA:F:\SoftWare\Anaconda\envs\mmrotate\lib\site-packages\mmdet\models\dense_heads\anchor_head.py:123: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead
warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 30/30, 0.9 task/s, elapsed: 33s, ETA: 0sTraceback (most recent call last):
File "tools/test.py", line 257, in
Reproduces the problem - command or script
python tools/test.py --config run_model/roi_trans_1/roi_trans_r50_fpn_1x_dota_le90.py --checkpoint run_model/roi_trans_1/latest.pth --eval mAP --show-dir work_dirs/roi_trans_1/result
Reproduces the problem - error message
TypeError: list indices must be integers or slices, not tuple
Additional information
Test and calculate the ap on your own dota data set.
Make your own dataset and make it into Dota format.
Hi @Yangzishy! This error is caused by the loading error of your dataset. Please check whether your dataset annotation format is consistent with the DOTA.
Hi @Yangzishy! This error is caused by the loading error of your dataset. Please check whether your dataset annotation format is consistent with the DOTA Hi @zytx121！ The format of my test data set is in Dota format, and it can detect the results of the test set, but it cannot calculate ap
Hi @Yangzishy! This error is caused by the loading error of your dataset. Please check whether your dataset annotation format is consistent with the DOTA.
Hi @zytx121 ! Please take a look at my reply. I need your help. Thank you
Hi @Yangzishy @jf0606, the maintenance of the old version has stopped. Welcome to the new version: https://github.com/open-mmlab/mmrotate/tree/1.x
Hi @Yangzishy! This error is caused by the loading error of your dataset. Please check whether your dataset annotation format is consistent with the DOTA.
Hi @zytx121 ! Please take a look at my reply. I need your help. Thank you
you can check you code in dotav1.py. I found it worked when i modify test=dict{ann_file=your path}
test=dict( type=dataset_type, ann_file=data_root + 'test/annfiles/', img_prefix=data_root + 'test/images/', pipeline=test_pipeline))
Hi @Yangzishy! This error is caused by the loading error of your dataset. Please check whether your dataset annotation format is consistent with the DOTA.
Hi @zytx121 ! Please take a look at my reply. I need your help. Thank you
you can check you code in dotav1.py. I found it worked when i modify test=dict{ann_file=your path}
test=dict( type=dataset_type, ann_file=data_root + 'test/annfiles/', img_prefix=data_root + 'test/images/', pipeline=test_pipeline))Hi @jf0606 ！Have you solved it？
Hi @Yangzishy @jf0606, the maintenance of the old version has stopped. Welcome to the new version: https://github.com/open-mmlab/mmrotate/tree/1.x
Hi @zytx121 ！Thank you for your answer. I'll try the new version
Hi @Yangzishy! This error is caused by the loading error of your dataset. Please check whether your dataset annotation format is consistent with the DOTA.
Hi @zytx121 ! Please take a look at my reply. I need your help. Thank you
you can check you code in dotav1.py. I found it worked when i modify test=dict{ann_file=your path}
test=dict( type=dataset_type, ann_file=data_root + 'test/annfiles/', img_prefix=data_root + 'test/images/', pipeline=test_pipeline))
Hi @jf0606 ！I have the same problem you mentioned, but it doesn't solve the problem. The same error still occurs
Same issue, when i try to find mAP on custom dataset but with the config file same as DOTA (with just classes being changed)
as @jf0606 pointed:
Hi @Yangzishy! This error is caused by the loading error of your dataset. Please check whether your dataset annotation format is consistent with the DOTA.
Hi @zytx121 ! Please take a look at my reply. I need your help. Thank you
you can check you code in dotav1.py. I found it worked when i modify test=dict{ann_file=your path}
test=dict( type=dataset_type, ann_file=data_root + 'test/annfiles/', img_prefix=data_root + 'test/images/', pipeline=test_pipeline))
I tried changing the images to annfiles directory but this gives a new error as below:
warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 66/66, 2.6 task/s, elapsed: 25s, ETA: 0sTraceback (most recent call last):
File "/home/user/manjunath/mmrotate/tools/test.py", line 257, in
Same issue, when i try to find mAP on custom dataset but with the config file same as DOTA (with just classes being changed)
as @jf0606 pointed:
Hi @Yangzishy! This error is caused by the loading error of your dataset. Please check whether your dataset annotation format is consistent with the DOTA.
Hi @zytx121 ! Please take a look at my reply. I need your help. Thank you
you can check you code in dotav1.py. I found it worked when i modify test=dict{ann_file=your path}
test=dict( type=dataset_type, ann_file=data_root + 'test/annfiles/', img_prefix=data_root + 'test/images/', pipeline=test_pipeline))I tried changing the images to annfiles directory but this gives a new error as below:
warnings.warn('DeprecationWarning: anchor_generator is deprecated, ' [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 66/66, 2.6 task/s, elapsed: 25s, ETA: 0sTraceback (most recent call last): File "/home/user/manjunath/mmrotate/tools/test.py", line 257, in main() File "/home/user/manjunat/mmrotate/tools/test.py", line 249, in main metric = dataset.evaluate(outputs, **eval_kwargs) File "/home/user/manjunath/mmrotate/mmrotate/datasets/dota.py", line 205, in evaluate mean_ap, _ = eval_rbbox_map( File "/home/user/manjunath/mmrotate/mmrotate/core/evaluation/eval_map.py", line 177, in eval_rbbox_map cls_dets, cls_gts, cls_gts_ignore = get_cls_results( File "/home/user/manjunath/mmrotate/mmrotate/core/evaluation/eval_map.py", line 114, in get_cls_results cls_gts.append(ann['bboxes'][gt_inds, :]) TypeError: list indices must be integers or slices, not tuple
Hi @jf0606 !Thank you for your response. I use mmrotate-1. x to test my dataset model with a map value
其实就是路径写错了，源代码dotav1.py“ test=dict( type=dataset_type, ann_file=data_root + 'test/images/', img_prefix=data_root + 'test/images/', pipeline=test_pipeline))” 把 ann_file=data_root + 'test/images/',改为 ann_file=data_root + 'test/annfiles/'
DOTA's test set has only images, no tags, that is, no annfiles, so eval map cannot be used on the test set. According to the document please submit assessment https://github.com/open-mmlab/mmrotate/blob/main/docs/en/get_started.md to generate compressed files online
