Nadav Elyahu comments

Repositories
Issues
Comments

Results 23 comments of


                                            Nadav Elyahu

z3 scaled_global_grad_norm: repalce get_global_norm with torch.norm

@loadams Oh, Sorry- I missed the last comment. Sure, yes we can add such UT that will cover it. But i cannot address it immediately. I will update this PR...

z3 scaled_global_grad_norm: repalce get_global_norm with torch.norm

@loadams / @tjruwase as request i make sure the regression was discussed here will be covered by a unit tests. I used TestZeroPartialOffloadConfigSweep, and added it gradient_clipping so it will...

z3 scaled_global_grad_norm: repalce get_global_norm with torch.norm

@loadams can you re-run workflows? i suspect that the failure are not related to this PR