renne444

INT8 Quantization Performance Issue with BERT-like Model

1

I am currently working on INT8 quantization for a **BERT-like** embedding model. In the last issue I raised, you mentioned that TensorRT does not currently support INT8 calibration for BERT-like...

[Bug]: Nvidia L20推理Qwen2.5 72B GPTQ-Int8模型不符合预期

8

### Model Series Qwen2 ### What are the models used? Qwen2.5-72B-Instruct-GPTQ-Int8 and Qwen2-72B-Instruct-GPTQ-Int8 ### What is the scenario where the problem happened? transformers ### Is this a known issue? -...