ao icon indicating copy to clipboard operation
ao copied to clipboard

Add Autoquant Cache

Open drisspg opened this issue 5 months ago • 1 comments

Summary

Today whenever a user runs autoquant, the AutoQuantCache gets populated with dtype + information for Linears seen within an arbitrary torch.nn.Module. This cache is not persistent. We should add a way to persist the benchmarking information across runs.

Details

  1. We likely want a similar paradigm to inductor: store cache to /tmp/torchaoautoquant_{user}.
  2. Provide a mechanism for overriding save location + whether it should be used

drisspg avatar Sep 09 '24 20:09 drisspg