Add manual synchronization function

[qbx2]

Hello. First of all, thank you for sharing code and experiment results. Reading the code, I found that the model will use fast weights to infer. According to LookAhead, fast weights (before synchronization) may perform worse than slow weights. By chance of (1-1/k) probability (80% when k=5), we will use unsynchronized fast weights to validate/test. Therefore, it should be better if we manually synchronize before evaluation.

[lessw2020]

Hi @qbx2 , Thanks for the feedback! I think adding a manual sync is a good idea, along with a manual clear weights (for restarting new training). I will try and add that later this week! Thanks Less

[qbx2]

Another minor issue: There is a typo N_sma_threshhold in ranger.py. It should be N_sma_threshold.