llama.cpp
llama.cpp copied to clipboard
ggml-org
Update tools.sh to use consolidated
Update to use the correct file name consolidated.*.pth. tools.sh is still looking for ggml-model-f16.bin* " skipping the file convertions. I wasn´t able to test this change because for some reason when building the docker image in windows it can find the shell.
The readme still says:
The easiest way to download the models, convert them to ggml and optimize them is with the --all-in-one command which includes the full docker image.
We should probable tell people exactly how to use downloaded files with docker, or this #493 will happen again and again.
I don't really have much experience with docker, but I think you'd use --volume for that.
The example commandlines are also broken with regards to string quoting, at least for me on Linux. Maybe @bernatvadell can help here?
I agree we should make it more clear, in the example: "docker run -v /llama/models:/models ghcr.io/ggerganov/llama.cpp:full --all-in-one "/models/" 7B" the models should be put in the folder: /llama/models and it can be updated to where the models is saved.
The commandlines are indeed broken, i noticed before. I dont know how to resolve that one but I think that should be covered in a different PR. From the testing I did the string seems to break for example: "Building a website can be done in 10 simple steps:" the main will only get the "Building" has an argument.
This appears to have the wrong type of line endings in .devops/tools.sh - gives me the following error: "standard_init_linux.go:228: exec user process caused: no such file or directory"
This might be an OS specific error. It can be avoided by setting the following before cloning the repo: git config --global core.autocrlf false
-v /llama/models
From where can I download the models, in order to populate that directory? Thanks.
