llama-recipes icon indicating copy to clipboard operation
llama-recipes copied to clipboard

Checkpoint feature via steps instead of epoch

Open mylesgoose opened this issue 4 months ago • 5 comments

🚀 The feature, motivation and pitch

at the moment the scritp only saves via epoch. for large data sets this is quite bad.

Alternatives

i crated an alternative here here

Additional context

the script will now save at the specified interval during the traning. and mark the files or folder according to the step and epoch. also it fixes some of the errors found in the original logic

mylesgoose avatar Oct 11 '24 03:10 mylesgoose