andrewor14

Add generic fake quantized linear for QAT

2

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #1020 **Summary:** This commit adds a generic fake quantized linear module to replace the uses of the existing more specific QAT linears....

Make module swap the main QAT flow again

1

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #1020 * __->__ #1019 **Summary:** Following https://github.com/pytorch/ao/issues/987, this commit makes module swap the main QAT flow today. We remove all tensor subclass fake...

Move and rename GranularityType -> Granularity

1

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #1020 * __->__ #1038 * #1037 Summary: Move GranularityType to quant_primitives.py to be consistent with other similar fields like MappingType and ZeroPointDomain. Test...

Make module swap the main QAT flow again

1

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #1020 * #1038 * __->__ #1037 Summary: Following https://github.com/pytorch/ao/issues/987, this commit makes module swap the main QAT flow today. We remove all tensor...

Fix QAT module swap BC

2

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #1058 Add back _module_swap_api.Int8DynActInt4WeightQATLinear. Differential Revision: [D64252460](https://our.internmc.facebook.com/intern/diff/D64252460)

Loss not going down for fine-tuning Llama3-8B on C4

7

I'm fine-tuning Llama3-8B on the C4 dataset (en subset) for 2000 steps using the `full_finetune_distributed` recipe. I find that the loss did not go down at all and the quantized...

Can we support outputting checkpoints directly in .pt format?

8

Today we need to do an extra conversion step according to this README: https://github.com/pytorch/torchtitan/blob/main/docs/checkpoint.md ``` python -m torch.distributed.checkpoint.format_utils dcp_to_torch outputs/checkpoint/step-100 /tmp/checkpoint.pt ``` I think we should **provide an option for...

Today, users have to do manual conversions between `.pth` and `.safetensors` formats before/after fine-tuning with torchtune. **Example 1: torchtitan -> torchtune -> HF transformers.** torchtitan outputs `.dcp`, which can be...

This issue trackers follow-up feature requests regarding QAT range learning after the initial PR: https://github.com/pytorch/ao/pull/2033 - [x] Do not learn zero points for symmetric quantization - [ ] Add option...

torch.export failed with tensor subclass, worked in 2.9.0

1

Exporting a model with tensor subclass led to the following error. This worked in 2.9.0 but not in the latest nightly: ``` RuntimeError: We found a fake tensor in the...