trl icon indicating copy to clipboard operation
trl copied to clipboard

Grad accumulation and memory bugfix

Open edbeeching opened this issue 2 years ago • 2 comments

  • Adds command line arg passing to sentiment and toxicity examples
  • Adds gradient accumulation as a command line arg #218
  • Fixes tensors not being detached from graph when stored in stats, leading to an overuse of memory.
  • Updates forward_batch_size -> minibatch size in a number of examples

By the way, I think there may be other places we use excessive memory due to storing attached tensors for too long. I will investigate further.

edbeeching avatar Mar 14 '23 12:03 edbeeching

I forgot to run style / quality. I am not on my dev machine at the moment. I will run this in an hour.

edbeeching avatar Mar 14 '23 12:03 edbeeching

The documentation is not available anymore as the PR was closed or merged.