fms-fsdp icon indicating copy to clipboard operation
fms-fsdp copied to clipboard

revert "raise Dynamo accumulated cache size limit"

Open lchu-ibm opened this issue 11 months ago • 0 comments

We recently added a commit to raise Dynamo accumulated cache size limit to make compile work with large models like 70b whose num_layer is greater than default limit (64): https://github.com/foundation-model-stack/fms-fsdp/pull/45#issuecomment-2002564455.

Now this number has been officially raised in PyTorch (related PR) from 64 to 256. So we can revert this commit as the new default is enough.

I am holding the revert change as the torch PR just got merged in today's nightly, and most env does not have this "fix" yet.

This is to be revisited later once that PR is picked by most env we have, and then we can revert it.

lchu-ibm avatar Mar 19 '24 16:03 lchu-ibm