ao
ao copied to clipboard
Add Autoquant Cache
Summary
Today whenever a user runs autoquant, the AutoQuantCache gets populated with dtype + information for Linears seen within an arbitrary torch.nn.Module. This cache is not persistent. We should add a way to persist the benchmarking information across runs.
Details
- We likely want a similar paradigm to inductor: store cache to /tmp/torchaoautoquant_{user}.
- Provide a mechanism for overriding save location + whether it should be used