text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Is there plan to support GPTQ?

Open zhch12121 opened this issue 2 years ago • 1 comments

Feature request

GPTQ is not supported yet in current version. Is there any timeline on that?

Motivation

quantization

Your contribution

can help with code if needed

zhch12121 avatar Jun 13 '23 03:06 zhch12121

Yes, GPT-Q will be supported once pr #438 is merged.

OlivierDehaene avatar Jun 13 '23 08:06 OlivierDehaene

Closing this issue since GPTQ is supported now.

danieldk avatar Jul 26 '24 13:07 danieldk