Loading state doesn't seem to be fully working

[tchaton]

To save : 'optimizer' : optimizer.state_dict()

optimizer.load_state_dict(checkpoint['optimizer'])

However, I have the impression restarting the training always bring the accuracy down and then it recovers.

Best, Thomas Chaton>