gleize
Results
2
comments of
gleize
I'm having the same issue when running FSDP + manual backward in a different setup. I was also able to reproduce the bug using the code provided above. Remark :...
@carlosgjs I found a workaround. Essentially, we need to exit the FSDP forward function so it changes its state from `FORWARD` to `BACKWARD_PRE`. So, I moved the `manual_backward` to the...