Robert Irvine
Results
3
issues of
Robert Irvine
Will it be possible to add support for GPTJ please? Any plans for this?
new model
Hi, I am trying to integrate AWQ into vLLM library. The current issue is that AWQ has worse throughput than the unquantised variant: I think it should at least match...
I find that the quantisation losses are higher for GPTJ than LLama which seems to stay pretty low. ``` 2023-06-20 19:05:19 INFO [auto_gptq.modeling._base] Quantizing attn.q_proj in layer 2/28... 2023-06-20 19:05:20...
bug