Can I save the quantized model to disk to avoid calling `torch.quantization.quantize_dynamic` each times?

[shy2052]

I've managed to run custom_whisper.py, but I'm thinking if we can save the quantization to disk, and let the whisper CLI (with the modified nn.Linear part) use it like other official models.

[frankiedrake]

Actually this part already exists in the code (here) It is: torch.save(model.state_dict(), path)

[frankiedrake]

IMPORTANT! What I have noticed is you won't be able to load whisper model out of this file, because it required some extra params. To save-> load the model correctly just use torch.save(model, path) and then torch.load(path) (omitting state_dict() func call)