open_lm icon indicating copy to clipboard operation
open_lm copied to clipboard

Published 20 hours ago • verified publisher icon mlfoundations

Add option to predownload data from s3 at the start of each checkpoint.

Open GeorgiosSmyrnis opened this issue 9 months ago • 0 comments

This adds an option that allows for predownloading of data at the start of each checkpoint to local storage. This helps with potential s3 errors.

GeorgiosSmyrnis avatar May 18 '24 19:05 GeorgiosSmyrnis