Forkoz

What does it sent when you put on verbose? The placeholder not getting replaced?

Nothing changed in bits&bytes.

I wonder if his changes can just be merged to GPTQ itself. That would make things easier. Some dataset py changes were only ones that alter GPTQ stuff. Unless the...

I get nan error from it when generating. The kernel is fine unless you have an older GPU.. like pre-pascal OSST works.

On older GPU it will NEVER work with int8 threshold at 6. But I get NaN error and not this error on my P6000. I am using the pre "fixed"...

I come here to tell you that the new accepted transformers is slow for me and I have no clue what is wrong on your cards and why mine works....

From reading their twitter.. I think this is some kind of pre-release so no readme or anything else. They are converting the 30b still.

TIL. This will solve the problem of git complaining if I do a pull. I also see those parameters in shared.py. When I was looking at that yesterday I noticed...

Lora 100% is supposed to make it deterministic: https://github.com/oobabooga/text-generation-webui/issues/419 If it is not then the lora isn't working.

Well 4 bit by itself is deterministic. 8/fp16 was not, unless you count producing a stream of unending garbage every time as deterministic. Turning off do_sample allows 8bit to generate...