65B Model failed to quantize

[wbhart]

After many days of getting ECONRESET errors so that downloading times out, I finally managed to get through all of the downloading for all the consolidated.0X.pth files without them cutting out (fortunately the current code doesn't re-download files that it has already downloaded fully, so after many attempts it got all of them).

But now it says that it failed to quantize.

I'm using the 0.1 release of dalai from a few days ago.

[andresdelcampo]

Try running this inside dalai\llama:

python3 download-pth.py models 65B

You should get up to consolidated.07.pth and all the same size. If not, delete the file that is partial and retry.

That worked for me :).

Then you can also run:

python3 convert-pth-to-ggml.py models\65B\ 1

Update: I just found out I also needed to run "./quantize.sh 65B" in a Git Bash

[wbhart]

Thanks. That might be the easier way to do this. I've never fully understood why some downloads are resumable when they cut out and some not.

[wbhart]

I've subsequently located all the consolidated.0x.pth files. Apparently they get put in ~/llama.cpp/models/65B not in ~/dalai/models/65B.

But I've checked all the files are there and all the same size and the quantize failure I reported happens in that state.

I'll see if deleting various things might fix the issue.

[tareaps]

When downloading can corrupt one of the files, if the download was interrupted, so you need to determine which file is damaged and download it again, if you can not determine, then download everything again. Were you able to make a 65B model with 4 bit quantization? If yes, could you upload it to some file sharing site?