Bug: listing models needs to filter out GGUF extension

[jamesbraza]

Starting with an empty models/ and a fresh restart of the LocalAI server:

> ls models
> curl http://localhost:8080/models/apply -H "Content-Type: application/json" -d '{
     "url": "github:go-skynet/model-gallery/base.yaml",
     "name": "llama2-chat",
     "files": [
        {
            "uri": "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGUF/resolve/main/llama-2-13b-chat.Q4_K_S.gguf",
            "sha256": "106d3b9c0a8e24217f588f2af44fce95ec8906c1ea92ca9391147ba29cc4d2a4",
            "filename": "llama-2-13b-chat.Q4_K_S.gguf"
        }
     ]
   }'
# ...
> curl http://localhost:8080/models
{"object":"list","data":[{"id":"llama2-chat","object":"model"},{"id":"llama-2-13b-chat.Q4_K_S.gguf","object":"model"}]}
> ls models
chat.tmpl                    completion.tmpl              llama-2-13b-chat.Q4_K_S.gguf llama2-chat.yaml

It seems the GGUF extension is not being filtered out by the listing models endpoint. It should be filtered out, I think only the files ending in .yaml should show up.

[localai-bot]

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

but.... I can also be funny or helpful :smile_cat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

I can see that the issue is related to the models/apply endpoint and the filtering of the GGUF extension. This is a bug in the LocalAI codebase and it should be fixed. I will report this issue to the maintainers of the project and suggest a fix. In the meantime, you can work around this issue by manually removing the GGUF extension from the model file name before applying the model using the models/apply endpoint.

Sources:

• https://localai.io/models/index.html
• https://github.com/go-skynet/LocalAI/tree/master/api/openai/image.go

[mudler]

cc @dave-gray101 as he was the latest touching that code area, that should be an easy fix right? leaving it up for grabs

[bearrito]

@mudler I haven't contributed to this project before but am interested. Could I grab this?

[lunamidori5]

@bearrito go for it!

[bearrito]

@lunamidori5 Quick question. I didn't see a guide on contributing. Is there a typical developer flow people are using? Namely, test strategies?

[mudler]

@mudler I haven't contributed to this project before but am interested. Could I grab this?

Sure, feel free to go ahead

@lunamidori5 Quick question. I didn't see a guide on contributing. Is there a typical developer flow people are using? Namely, test strategies?

we have https://github.com/mudler/LocalAI/blob/master/CONTRIBUTING.md but it is out of sync.

For testing - we have quite many tests. I'd suggest you to use the newer aio tests which covers a smaller area but are easier to run locally:

# Build the LocalAI docker image
make DOCKER_IMAGE=local-ai docker
# Build the corresponding AIO image
BASE_IMAGE=local-ai DOCKER_AIO_IMAGE=local-ai-aio:test make docker-aio
# Run the AIO e2e tests
LOCALAI_IMAGE_TAG=test LOCALAI_IMAGE=local-ai-aio make run-e2e-aio

After you get green tests there you should be good to open up a PR and get all the tests against your changes