Cannot reproduce other results in Table 3.

[Johnzhangt]

Hi Author, Thanks for sharing the wonderful code. I run your code with the default setting for three times with the target dataset "sketch". The average results for three times are 78.07+-1.28, which is very close to the results reported in your paper (78.07+-1.43).

However, I cannot reproduce other results. For example, I run three times with the target dataset "Cartoon". The result is 80.24, 79.69, 80.97. The average result is 80.28+-0.52, which is lower than your paper. I also found similar issues with other target datasets, such as Photo and Art painting. The only thing I change is the dataset. I did not change any other parameters.

May I know what is the reason here? How can I reproduce other results in Table 3? Many thanks for your reply!

[Johnzhangt]

how many runs have you conducted for each setting? Are there any specific parameters set for these experiments?

[emma-sjwang]

The results reported in the paper are the average of the first three runs. Before release this code, we also have reproduced the method with this code which results are slightly higher than the results reported in the paper.

I think the problem may be the different initialization parameters or GPU version or cuda version?

Our method works well on TITAN Xp and GeForce RTX 2080 Ti with cuda 10.2.

The packages used are listed as follows for your reference and check.

Name Version Build Channel _libgcc_mutex 0.1 main
absl-py 0.9.0 pypi_0 pypi blas 1.0 mkl
ca-certificates 2020.6.24 0
cachetools 4.1.1 pypi_0 pypi certifi 2020.6.20 py36_0
chardet 3.0.4 pypi_0 pypi cudatoolkit 9.2 0
freetype 2.10.2 h5ab3b9f_0
google-auth 1.18.0 pypi_0 pypi google-auth-oauthlib 0.4.1 pypi_0 pypi grpcio 1.30.0 pypi_0 pypi idna 2.10 pypi_0 pypi importlib-metadata 1.7.0 pypi_0 pypi intel-openmp 2020.1 217
jpeg 9b h024ee3a_2
lcms2 2.11 h396b838_0
libedit 3.1.20191231 h14c3975_1
libffi 3.2.1 hd88cf55_4
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0
libpng 1.6.37 hbc83047_0
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.1.0 h2733197_1
lz4-c 1.9.2 he6710b0_0
markdown 3.2.2 pypi_0 pypi mkl 2020.1 217
mkl-service 2.3.0 py36he904b0f_0
mkl_fft 1.1.0 py36h23d657b_0
mkl_random 1.1.1 py36h0573a6f_0
ncurses 6.2 he6710b0_1
ninja 1.9.0 py36hfd86e86_0
numpy 1.18.5 py36ha1c710e_0
numpy-base 1.18.5 py36hde5b4d6_0
oauthlib 3.1.0 pypi_0 pypi olefile 0.46 py_0
openssl 1.1.1g h7b6447c_0
pillow 7.2.0 py36hb39fc2d_0
pip 20.1.1 py36_1
protobuf 3.12.2 pypi_0 pypi pyasn1 0.4.8 pypi_0 pypi pyasn1-modules 0.2.8 pypi_0 pypi python 3.6.8 h0371630_0
pytorch 1.4.0 py3.6_cuda9.2.148_cudnn7.6.3_0 pytorch pyyaml 5.1 pypi_0 pypi readline 7.0 h7b6447c_5
requests 2.24.0 pypi_0 pypi requests-oauthlib 1.3.0 pypi_0 pypi rsa 4.6 pypi_0 pypi setuptools 47.3.1 py36_0
six 1.15.0 py_0
sqlite 3.32.3 h62c20be_0
tensorboard 2.1.0 pypi_0 pypi tk 8.6.10 hbc83047_0
torchvision 0.5.0 py36_cu92 pytorch urllib3 1.25.9 pypi_0 pypi werkzeug 1.0.1 pypi_0 pypi wheel 0.34.2 py36_0
xz 5.2.5 h7b6447c_0
zipp 3.1.0 pypi_0 pypi zlib 1.2.11 h7b6447c_3
zstd 1.4.4 h0b5b093_3

[Johnzhangt]

What is your input size for Alexnet in Table 2?