Tal Almagor comments

Repositories
Issues
Comments

Results 2 comments of


Tal Almagor

Support for jupyter notebooks on VSCode

TabNine will not be able to work with the jupyter extension [custom editor](https://github.com/microsoft/vscode-jupyter/blob/55ea387553812a3ca7c7b15bd1984a485fe37754/package.json#L1349). But good news - it does seem to work great with the native editor API which is...

Model doesn't learn with fp16.enabled = true: [Rank 0] Overflow detected. Skipping step. Attempted loss scale: 1, reducing to 1

encountered roughly the same, but the problem was "nan gradients" caused by layer norm, or more specifically something similar to - https://github.com/pytorch/pytorch/issues/41527. and with fairseq - https://github.com/pytorch/fairseq/blob/fcca32258c8e8bcc9f9890bf4714fa2f96b6b3e1/fairseq/modules/layer_norm.py#L38