The functions example no longer works

[mtharrison]

LocalAI version: master

Environment, CPU architecture, OS, and Version: Darwin M-YPDW3RKWDM 22.6.0 Darwin Kernel Version 22.6.0: Wed Jul 5 22:22:05 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6000 arm64

Describe the bug The example at https://github.com/go-skynet/LocalAI/tree/master/examples/functions does not work whether in Docker or with a native build. I tried to load the same model with llama.cpp and I think it's because it's a gmml model (which I believe is no longer supported?). When I ran `convert-llama-ggml-to-gguf.py on the downloaded model I can run it with llama.cpp but unsure how to override the model file in local-ai.

To Reproduce Pull master, build local-ai and run the functions example.

I'm happy to make a PR if there's a substitute model that you can recommend. I don't know which models will work with the functions example. I tried a couple of gguf models with no success (output no function).

[mtharrison]

Just tested with this model and it seems to work well: https://huggingface.co/TheBloke/Llama-2-7B-32K-Instruct-GGUF/blob/main/llama-2-7b-32k-instruct.Q5_K_M.gguf

Can I help adding to model gallery and updating functions example?

[mudler]

Just tested with this model and it seems to work well: https://huggingface.co/TheBloke/Llama-2-7B-32K-Instruct-GGUF/blob/main/llama-2-7b-32k-instruct.Q5_K_M.gguf

Can I help adding to model gallery and updating functions example?

Yes, sure, feel free to help - this is an open source project!

[MLTQ]

The functions example isn't working for me either, using exactly the model you linked. Always returns empty with this:

Traceback (most recent call last):
  File "/home/m/.config/JetBrains/PyCharm2023.2/scratches/scratch_1.py", line 82, in <module>
    print(run_conversation())
          ^^^^^^^^^^^^^^^^^^
  File "/home/m/.config/JetBrains/PyCharm2023.2/scratches/scratch_1.py", line 59, in run_conversation
    fuction_to_call = available_functions[function_name]
                      ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
KeyError: None

Any idea why?

[mtharrison]

@MLTQ I think we need to use a gguf model. I have a PR open here: https://github.com/go-skynet/model-gallery/pull/27

I can provide some instructions for getting it to work in the meantime if you need.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.