Omri Algazi

timestamp check on get_snapshot

3

Hi everyone, thanks for this great contribution! I keep getting a False response from this check which made me wonder why is check even needed? ``` request_time = time.time() for...

Out of memory with RTX3090

4

Hi, I'm trying to train gpt2xl, but keep getting OOM, even when I set batch size to 1 and gradient_accumulation to 8\16\512, contigous_gradients false and allgather_bucket_size \ reduce_bucket_size 2e2. I...