Add manual synchronization function

[qbx2]

Hello. First of all, thank you for sharing code and experiment results.
Reading the code, I found that the model will use fast weights to infer. According to LookAhead, fast weights (before synchronization) may perform worse than slow weights. By chance of (1-1/k) probability (80% when k=5), we will use unsynchronized fast weights to validate/test. Therefore, it should be better if we manually synchronize before evaluation.

[lessw2020]

Hi @qbx2 ,
Thanks for the feedback!
I think adding a manual sync is a good idea, along with a manual clear weights (for restarting new training).
I will try and add that later this week!
Thanks
Less

[qbx2]

Another minor issue: There is a typo N_sma_threshhold in ranger.py. It should be N_sma_threshold.