DanielWe2

cpp headers not in target_include_directories

1

The cpp headers are not added to the target_include_directories. Without that it is hard to use CRoaring from a local subdirectory from C++ without install. (it can not find the...

Questions about group size

6

From the research paper and the tables in the readme it looks like that group-size 64 is very effective in improving the quality of the models. Most noticable in the...

I would like to know how the Winogrande results in the paper were achieved: What prompt and parameters were used?

Script to execute Winogrande test

1

Small script to execute WinoGrande tests See details in the README

Bad results for WinoGrande - more testers searched

1

Facebook published posted expected results for the WinoGrande test with a score of 70 for the 7B model. I wrote a small script see #40 that fetches the dataset from...

Needs more VRAM than normal GPTQ CUDA version?

3

Thanks, I wanted to try your triton version. But I only have 8 GB RAM. The GPTQ Cuda versions works (7B model). Your version (the ppl script) crashes with CUDA...