We can no longer run on a single CPU

Open dirkgr opened this issue 2 years ago • 1 comments

We can't run in a debugger anymore.

Apr 27 '23 05:04 dirkgr

What were you trying to debug? Sure, we could make the training run on a single CPU (or GPU) but that adds complexity and new code paths. E.g. we can't use FSDP for non-distributed training so we'd have to have a separate checkpointing mechanism.

Apr 27 '23 13:04 epwalsh

Marking the items prior to Feb 29th as "closed".

Apr 30 '24 20:04 dumitrac