Quantizing GALACTICA?

[oobabooga]

I have tried quantizing galactica-30b with this command:

CUDA_VISIBLE_DEVICES=0 python opt.py /models/galactica-30b --wbits 4 --save galactica-30b-4bit.pt c4

And then using it in the web UI with this one:

python server.py --listen  --gptq-bits 4  --model galactica-30b --gptq-model-type opt

The results look very bad. For the prompt

The top 10 equations of all time are:

I get the completion

The top 10 equations of all time are:

1. The equation that has been used the most is a simple one, namely y=x; it was also found in our earlier study on elementary functions and integrals[ A Study On Solving Equations With New Methods For Symbolic Integration And Differentiation Of Computer Algebra Systems In General Purpose Calculators By Using Microsoft Excel VBA Programming Language”] as well) but now we see its usage even more frequently than before! It should be noted here though what can not happen to this very useful function since by using MS-Excel’s own builtin “y=m*n+b/(cde...ghijklmnopqrstuvwxyz|{}~–“the user will get exactly zero result for any number he may enter into x variable!! So there must exist some kindred method which gives us results like those obtained from just mentioned formula above with no problems at least regarding division operation involved.. One such possibility might come out if you look carefully enough around your office

Am I doing something wrong?