The reproduced results are relatively poor

[liuxingyu123]

Hello, the pretrained model I use for vit is vit_ large_ patch16_ 384, but the results of the first stage of training are ODS: 0.74116, OIS: 0.761012, AP: 0.736421. In the paper, ODS: 0.817, OIS: 0.835, AP: 0.867, the parameter of code has not changed. What is the problem？

[liuxingyu123]

Is it a problem of environment configuration? My environment is python3.8，pytorch1.7.0 cuda11.0 mmcv1.2.3.

[MengyangPu]

Python: 3.7.7 [GCC 7.3.0] CUDA available: True GPU: Tesla V100-PCIE-32GB GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 PyTorch: 1.6.0 PyTorch compiling details: PyTorch built with:

GCC 7.3 C++ Version: 201402 Intel(R) Math Kernel Library Version 2020.0.1 Product Build 20200208 for Intel(R) 64 architecture applications Intel(R) MKL-DNN v1.5.0 (Git Hash e2ac1fac44c5078ca927cb9b90e1b3066a0b2ed0) OpenMP 201511 (a.k.a. OpenMP 4.5) NNPACK is enabled CPU capability usage: AVX2 CUDA Runtime 10.1 NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37 CuDNN 7.6.3 Magma 2.5.2 TorchVision: 0.7.0 OpenCV: 4.5.1 MMCV: 1.2.4 MMCV Compiler: GCC 7.3 MMCV CUDA Compiler: 10.1 MMSegmentation: 0.6.0+

[MengyangPu]

We fixed and updated the codes, please redownload and run, thanks~

[yun-liu]

Have anyone reproduced the results? Could you share how to do it?

[hhqweasd]

I reproduced the first stage of training with vit large_ patch16_ 384_ backbone, the ODS and OIS are 0.813 and 0.830 respectively on the tesing set of BSDS at iter_10k, which are similar to the reported results: ODS: 0.817, OIS: 0.835

---pip envs--- addict 2.4.0 appdirs 1.4.4 certifi 2022.12.7 cityscapesScripts 2.2.1 clip 1.0 coloredlogs 15.0.1 contourpy 1.0.7 cycler 0.11.0 fonttools 4.39.3 ftfy 6.1.1 future 0.18.3 h5py 3.8.0 humanfriendly 10.0 importlib-resources 5.12.0 kiwisolver 1.4.4 matplotlib 3.7.1 mmcv-full 1.2.2 mmsegmentation 0.6.0 /root/EDTER-main numpy 1.24.2 opencv-python 4.7.0.72 packaging 23.0 Pillow 9.5.0 pip 23.0.1 pyparsing 3.0.9 pyquaternion 0.9.9 python-dateutil 2.8.2 PyYAML 6.0 regex 2023.3.23 scipy 1.10.1 setuptools 65.6.3 six 1.16.0 torch 1.6.0+cu101 torchvision 0.7.0+cu101 tqdm 4.65.0 typing 3.7.4.3 wcwidth 0.2.6 wheel 0.38.4 yapf 0.32.0 zipp 3.15.0

---conda envs--- _libgcc_mutex 0.1 main https://mirrors.aliyun.com/anaconda/pkgs/main _openmp_mutex 5.1 1_gnu https://mirrors.aliyun.com/anaconda/pkgs/main addict 2.4.0 pypi_0 pypi appdirs 1.4.4 pypi_0 pypi ca-certificates 2023.01.10 h06a4308_0 https://mirrors.aliyun.com/anaconda/pkgs/main certifi 2022.12.7 py38h06a4308_0 https://mirrors.aliyun.com/anaconda/pkgs/main cityscapesscripts 2.2.1 pypi_0 pypi clip 1.0 pypi_0 pypi coloredlogs 15.0.1 pypi_0 pypi contourpy 1.0.7 pypi_0 pypi cycler 0.11.0 pypi_0 pypi fonttools 4.39.3 pypi_0 pypi ftfy 6.1.1 pypi_0 pypi future 0.18.3 pypi_0 pypi h5py 3.8.0 pypi_0 pypi humanfriendly 10.0 pypi_0 pypi importlib-resources 5.12.0 pypi_0 pypi kiwisolver 1.4.4 pypi_0 pypi ld_impl_linux-64 2.38 h1181459_1 https://mirrors.aliyun.com/anaconda/pkgs/main libffi 3.4.2 h6a678d5_6 https://mirrors.aliyun.com/anaconda/pkgs/main libgcc-ng 11.2.0 h1234567_1 https://mirrors.aliyun.com/anaconda/pkgs/main libgomp 11.2.0 h1234567_1 https://mirrors.aliyun.com/anaconda/pkgs/main libstdcxx-ng 11.2.0 h1234567_1 https://mirrors.aliyun.com/anaconda/pkgs/main matplotlib 3.7.1 pypi_0 pypi mmcv-full 1.2.2 pypi_0 pypi mmsegmentation 0.6.0 dev_0 ncurses 6.4 h6a678d5_0 https://mirrors.aliyun.com/anaconda/pkgs/main numpy 1.24.2 pypi_0 pypi opencv-python 4.7.0.72 pypi_0 pypi openssl 1.1.1t h7f8727e_0 https://mirrors.aliyun.com/anaconda/pkgs/main packaging 23.0 pypi_0 pypi pillow 9.5.0 pypi_0 pypi pip 23.0.1 py38h06a4308_0 https://mirrors.aliyun.com/anaconda/pkgs/main pyparsing 3.0.9 pypi_0 pypi pyquaternion 0.9.9 pypi_0 pypi python 3.8.16 h7a1cb2a_3 https://mirrors.aliyun.com/anaconda/pkgs/main python-dateutil 2.8.2 pypi_0 pypi pyyaml 6.0 pypi_0 pypi readline 8.2 h5eee18b_0 https://mirrors.aliyun.com/anaconda/pkgs/main regex 2023.3.23 pypi_0 pypi scipy 1.10.1 pypi_0 pypi setuptools 65.6.3 py38h06a4308_0 https://mirrors.aliyun.com/anaconda/pkgs/main six 1.16.0 pypi_0 pypi sqlite 3.41.1 h5eee18b_0 https://mirrors.aliyun.com/anaconda/pkgs/main tk 8.6.12 h1ccaba5_0 https://mirrors.aliyun.com/anaconda/pkgs/main torch 1.6.0+cu101 pypi_0 pypi torchvision 0.7.0+cu101 pypi_0 pypi tqdm 4.65.0 pypi_0 pypi typing 3.7.4.3 pypi_0 pypi wcwidth 0.2.6 pypi_0 pypi wheel 0.38.4 py38h06a4308_0 https://mirrors.aliyun.com/anaconda/pkgs/main xz 5.2.10 h5eee18b_1 https://mirrors.aliyun.com/anaconda/pkgs/main yapf 0.32.0 pypi_0 pypi zipp 3.15.0 pypi_0 pypi zlib 1.2.13 h5eee18b_0 https://mirrors.aliyun.com/anaconda/pkgs/main

---gpustat--- [0] Tesla V100-PCIE-32GB | 70'C, 94 % | 25398 / 32768 MB | [1] Tesla V100-PCIE-32GB | 68'C, 88 % | 25398 / 32768 MB | Note that all the GPUs must be on the same machine, not distributed.

---samples_per_gpu--- samples_per_gpu=4

---training command--- ./tools/dist_train.sh configs/bsds/EDTER_BIMLA_320x320_80k_bsds_bs_8.py 2

Feel free to contact me if you need more information. @liuxingyu123 @yun-liu

---update--- Note that all the GPUs must be on the same machine, not distributed. iter_20k: ODS=0.816 OIS=0.832 AP=0.865.

[MengyangPu]

We update the reproduced results in https://github.com/MengyangPu/EDTER#4-the-original-results-vs-the-reproduced-results and provide all results.