Reproducibility issue with CuDNN backend

[songyouwei]

https://github.com/namisan/mt-dnn/blob/25c3bd1cfee2db14c7e8dbec4a369acaa95b15f6/data_utils/utils.py#L25

When running on the CuDNN backend, there might be reproducibility issue. According to pytorch document, two further options must be set:

torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

[namisan]

Thanks pointing it out!! Does this work on pytorch version 0.4x? We will test it soon.

[songyouwei]

I haven't tested it on pytorch version 0.4x.