auto-round icon indicating copy to clipboard operation
auto-round copied to clipboard

Published 20 hours ago • verified publisher icon intel

WIP: fix compat with latest autogptq and use meta region to store auto-round properties

Open Qubitium opened this issue 2 months ago • 10 comments

Reason for PR:

  1. Fix compat with latest autogptq
  2. Store autoround fingerprint/version using meta_set_quantizer(name, version) api
  3. Store autoround specific parameters, unrelated to actual autogptq inference/quantization, into meta region via meta_set api
  4. add tqdm progress to quantization so user get good estimation of iter/s and remaining time

Pending merge/changes to https://github.com/AutoGPTQ/AutoGPTQ/pull/640

  • [x] Tested quant with autogptq inference with sym=True and False.

Qubitium avatar Apr 24 '24 17:04 Qubitium