mmrotate
mmrotate copied to clipboard
open-mmlab
[Bug] DOTA Evaluation Results Feedback of Task1 is obviously lower than the official result
Prerequisite
- [X] I have searched Issues and Discussions but cannot get the expected help.
- [X] I have read the FAQ documentation but cannot get the expected help.
- [X] The bug has not been fixed in the latest version (master) or latest version (1.x).
Task
I'm using the official example scripts/configs for the officially supported tasks/models/datasets.
Branch
master branch https://github.com/open-mmlab/mmrotate
Environment
sys.platform: linux Python: 3.7.0 (default, Oct 9 2018, 10:31:47) [GCC 7.3.0] CUDA available: True GPU 0,1,2,3,4,5,6,7: NVIDIA GeForce RTX 3090 CUDA_HOME: /usr/local/cuda-11.3 NVCC: Cuda compilation tools, release 11.3, V11.3.109 GCC: gcc (Ubuntu 5.5.0-12ubuntu1) 5.5.0 20171010 PyTorch: 1.11.0+cu113 PyTorch compiling details: PyTorch built with:
- GCC 7.3
- C++ Version: 201402
- Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.5.2 (Git Hash a9302535553c73243c632ad3c4c80beec3d19a1e)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.3
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
- CuDNN 8.2
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.11.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,
TorchVision: 0.12.0+cu113 OpenCV: 4.9.0 MMCV: 1.5.3 MMCV Compiler: GCC 7.3 MMCV CUDA Compiler: 11.3 MMRotate: 0.3.4+9ea1aee
Reproduces the problem - code sample
python tools/train.py configs/rotated_fcos/rotated_fcos_r50_fpn_1x_dota_le90.py
Reproduces the problem - command or script
python tools/test.py configs/rotated_fcos/rotated_fcos_r50_fpn_1x_dota_le90.py work_dirs/rotated_fcos_r50_fpn_1x_dota_le90/latest.pth --format-only --eval-options submission_dir=task1
Reproduces the problem - error message
The detection accuracy of FCOS on the DOTA dataset is much lower than the model provided by model zoo
Additional information
This is your evaluation result for task 1 (VOC metrics):
mAP: 0.42395437677254383 ap of each class: plane:0.5423819423982649, baseball-diamond:0.5285280870317047, bridge:0.20667867900155412, ground-track-field:0.4569022199523975, small-vehicle:0.22461535065045377, large-vehicle:0.08442215012610903, ship:0.17547907678219063, tennis-court:0.6117272119996067, basketball-court:0.6934436730389884, storage-tank:0.8345323594333807, soccer-ball-field:0.36092004275259465, roundabout:0.6534027091402538, harbor:0.247273973001902, swimming-pool:0.49025477053589234, helicopter:0.24875340574286495
i have checked the config and confirmed the file is same as the official config, and the method of split is same, i attached my log file 20240321_214044.log
Maybe you didn't use the augumented data.
By the way we encountered an issue with the DOTA V1.0 evaluation server, we can't link to the below links.
Maybe you didn't use the augumented data. By the way we encountered an issue with the DOTA V1.0 evaluation server, we can't link to the below links.
i have not meet this issue , and how to use the augumented data?
Like muti-scale training and some test tricks. Or you can paste your config here for more details.
Our team has been experiencing issues with connecting to the evaluation server as i mentioned above. I would like to know if you can send me the evaluation URL or give me some kind of help with connecting.
------------------ 原始邮件 ------------------ 发件人: "open-mmlab/mmrotate" @.>; 发送时间: 2024年3月23日(星期六) 中午1:45 @.>; @.@.>; 主题: Re: [open-mmlab/mmrotate] [Bug] DOTA Evaluation Results Feedback of Task1 is obviously lower than the official result (Issue #1013)
Maybe you didn't use the augumented data. By the way we encountered an issue with the DOTA V1.0 evaluation server, we can't link to the below links.
i have not meet this issue , and how to use the augumented data?
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>
@jimuIee Have you solved this problem? I have the same problem as you, could you please help me?
