MAML-Pytorch icon indicating copy to clipboard operation
MAML-Pytorch copied to clipboard

Result is not deterministic because of iterating over Python dictionary

Open ShawnLixx opened this issue 5 years ago • 3 comments

https://github.com/dragen1860/MAML-Pytorch/blob/98a00d41724c133bd29619a2fb2cc46dd128a368/MiniImagenet.py#L81 should be changed to OrderedDict.

Result can't be reproduced because the use of dictionary. Iterating over dict causes randomness. https://github.com/dragen1860/MAML-Pytorch/blob/98a00d41724c133bd29619a2fb2cc46dd128a368/MiniImagenet.py#L68

ShawnLixx avatar Oct 18 '19 08:10 ShawnLixx

Also, random.seed should be called in the main function of training scripts.

ShawnLixx avatar Oct 22 '19 08:10 ShawnLixx

This is only a problem if you're using Python < 3.6 3.6 onwards, insertion order is maintained for the default dictionary as well.

https://stackoverflow.com/questions/1867861/how-to-keep-keys-values-in-same-order-as-declared

But still I do agree that it's more backward compatible to change it to OrderedDict as @ShawnLixx suggests

semin-park avatar Dec 12 '19 08:12 semin-park

Even OrderedDict and random.seed are used, the results are still not reproducible.

atseng17 avatar Dec 11 '20 21:12 atseng17