Quantizer produces 0 byte file and release folder is empty.

[pdavis68]

I get this error when I run: npx dalai llama install 7B in windows..

./quantize : The term './quantize' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:96
+ ... sole]::InputEncoding=[System.Text.Encoding]::UTF8; ./quantize F:\llm\ ...
+                                                        ~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (./quantize:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

I manually reran the command as administrator from the "build\bin\Release" folder (not to be confused with the "build\Release" folder, which is empty.

F:\LLM\llama\llama\build\bin\Release>quantize F:\llm\llama\llama\models\7B\ggml-model-f16.bin F:\llm\llama\llama\models\7B\ggml-model-q4_0.bin 2
llama_model_quantize: loading model from 'F:\llm\llama\llama\models\7B\ggml-model-f16.bin'
llama_model_quantize: n_vocab = 32000
llama_model_quantize: n_ctx   = 512
llama_model_quantize: n_embd  = 4096
llama_model_quantize: n_mult  = 256
llama_model_quantize: n_head  = 32
llama_model_quantize: n_layer = 32
llama_model_quantize: f16     = 1

but the ggml-model-q4_0.bin.2 file is 0 bytes in length.

When I run npx dalai serve and go to the web page, whenever I give it a prompt and click "GO" it shows busy and in the console it's running main and just hangs, but probably because it's trying to run it from the build\release folder, which is empty.

I also got 13B and same issue with quantizer and, unlike 7B, it didn't show up in the list of models when I ran npx dalai server

I'm running under windows. I assume that's obvious from the file paths.

[pdavis68]

I ran the quantize.py script as so: python ..\..\..\quantize.py -m ..\..\..\models 7B

This showed the same output as before except it ended (quickly) with: "An error occurred while trying to quanitze the models."

So this, I think is my biggest problem at this point. I tried downloading the latest alpaca-win from alpaca.cpp and run the Quantize in that and it tells me the model is invalid. So I'm stumped at this point.

[pdavis68]

Looking at the quantize.cpp it looks like it's looking for a -f32 file, not a -f16 file, so I manually ran convert-pth-to-ggml with an ftype of 0 and produced a -f32 file, but it still dropped out.

I then did an echo %errorlevel" and it gives me -1073741795 which is STATUS_ILLEGAL_INSTRUCTION

I'm running this all as admin, if that matters.

I tried rerunning quantize on the -f16 file as well and got the same error level, so it appears something is wrong with my quantize.exe

[xasx]

I also get the error above regarding the quantize.exe not being in the correct folder. How can this be fixed?