Kunal Chakrabarty comments

Repositories
Issues
Comments

Results 3 comments of


                                            Kunal Chakrabarty

List of Usability Changes

I see a lot of warnings of the following nature 2022-07-30 01:29:25 | WARNING | root | Rolled back to use the default process group for the reduce scatter operation...

Make epoch mean epoch again (in streaming language modeling)

Is the intent of this issue to just rename epoch into something more meaningful and intuitive like shard_index?

--arch transformer_lm and --tensor-init-on-gpu are incompatible

Is the expectation of this task to have a check that prevents setting both these flags? Or is it to explore the underlying issue that causes it?