flycser

fp16 bug about grad_norm computation (resulting in min loss scale reached)

3

## 🐛 Bug [FloatingPointError: Minimum loss scale reached (0.0001). ](https://github.com/facebookresearch/fairseq/issues/1529#issuecomment-567955547) I met this assertation often when do training in fp16. I found it may be avoided by modifying the way...

Hydra can not recognize "--local_rank=0" argument

3

## 🐛 Bug When I ran a model in a distributed model(2 nodes, each node with 2 GPUs) via hydra_train.py. the hydra can not accept arguments starting with "--", while...