torchtune icon indicating copy to clipboard operation
torchtune copied to clipboard

Update QAT recipe to match full_finetune_distributed

Open andrewor14 opened this issue 6 months ago • 1 comments

Summary: Many changes have gone into the full_finetune_distributed recipe but were not reflected on the equivalent qat_distributed recipe. This commit brings the latter up to date, adding features like FSDP2, configurable FSDP sharding, and profiling.

Test Plan: pytest -m integration_test tests/recipes/test_qat_distributed.py

Reviewers: ebsmothers, kartikayk, RdoubleA

Subscribers: ebsmothers, kartikayk, RdoubleA

andrewor14 avatar Aug 13 '24 23:08 andrewor14