How to Qauntize Wav2vec model?

[mehrdad78]

hi there. I want to quantize my wav2vec model .u used dynamic_quantize but i t did not help and my model get slower then before. how should i that? i search docs and issues but did not find any useful content. i think probably , i should Dequantize _k , q , v ._ but how can do that?

1. search the issues.
2. search the docs.

model_int8 = torch.quantization.quantize_dynamic(
  model,
  {torch.nn.Linear},
   dtype=torch.qint8,
   inplace=True
)

• transformer
• PyTorch
• Google Colab