Aadyot Bhatnagar

icon indicating an email [email protected]

icon company @Profluent-AI icon location Berkeley, CA icon location ML scientist working on sequence generation problems in biology. Previously worked at Salesforce on time series, uncertainty estimation, and speech recognition.

Results 24 comments of Aadyot Bhatnagar

Add option to accumulate train loss over tokens.

@dakinggg I've added a unit test that requires sample-based and token-based weighting result in different outcomes when padding is present.

Add option to accumulate train loss over tokens.

@dakinggg after some further thought, I made one additional change to the composer code. Currently, the total loss is just averaged across all ranks, since the trainer assumes that all...

Add option to accumulate train loss over tokens.

Thanks @mvpatel2000. For the time being, I've implemented these changes by overriding the `Trainer` class in our local repo, so we will be okay for now. Happy to get further...

Add option to accumulate train loss over tokens.

@dakinggg bumping this PR for review.