Can not reproduce the results in the paper when training from scratch

[florianlanger]

Hi Manuel, thanks for your work. I am having trouble to reproduce the retrieval accuracies (or ranking accuracies) quoted in the paper and in issue #4. I am training the network from scratch with the provided code (without any modifications) on the provided preprocessed data. The results are so far off that it seems like there must be a bug somewhere.

I get the following total retrieval accuracies after the respective number of train steps. Number steps: 2k , 25 k, 50 k , 75 k , 100 k Orig settings repo Retrieval accuracy: 0.03, 0.09, 0.10, 0.08, 0.03 Mod settings Retrieval accuracy: 0.01 , 0.03, 0.03, 0.05 , 0.05

For the mod settings in line with the paper I set the triplet margin to 0.2 (as opposed to 0.01) and change the learning rate scheduler to update every 20 k steps as opposed to 40 k steps. As far as I can tell all other settings in the repo match the information provided in the paper.

The result quoted in the paper is 0.43 and in issue #4 it is 0.42. I know this is fairly old code but would be amazing if either you or @TimFelixBeyer could look into this. Thanks a lot! Florian

[xheon]

See email. I'll leave it open until we have resolved it.