torchtune
torchtune copied to clipboard
Update QAT recipe to match full_finetune_distributed
Summary: Many changes have gone into the full_finetune_distributed recipe but were not reflected on the equivalent qat_distributed recipe. This commit brings the latter up to date, adding features like FSDP2, configurable FSDP sharding, and profiling.
Test Plan: pytest -m integration_test tests/recipes/test_qat_distributed.py
Reviewers: ebsmothers, kartikayk, RdoubleA
Subscribers: ebsmothers, kartikayk, RdoubleA