Sigbjørn Skjæret

@Rocketknight1 This looks great, however I think it might be worth making the `tools` parameter identical to [OpenAI](https://platform.openai.com/docs/api-reference/chat/create#chat-create-tools), ie. move the schema down a level (inside `function`) and have `type`...

> Hi @CISC, yes, we considered that! We felt that it just added extra complexity, though, and noticed that Anthropic skips that extra layer for tool-use in Claude too. I...

> @CISC Are there any examples of 'tools' that aren't functions out there? I'm curious about what those would look like! Yes, in OpenAI [Assistants](https://platform.openai.com/docs/api-reference/assistants/createAssistant#assistants-createassistant-tools) API. You can also easily...

@Rocketknight1 Don't forget to add the `tool` role to the template PRs you are submitting so that it's possible to do the full `user`->`tool_calls`->`tool`->`assistant` round-trip to get a natural response....

> Curious about the choice to not to follow OpenAI convention, with `tool_calls` being a separate role from `assistant`. @jenkspt Not sure what you mean? `tool_calls` is part of the...

> While it seems this PR has a different design (I'm looking at this: https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1/discussions/33) > > ```json > {"role": "assistant", "content": "..."} > {"role": "tool_calls", "content": "..."} > {"role":...

> Outputs from tools are sent as messages with `message['role'] == "tool_results"` @Rocketknight1 Any reason for not going with the `tool` role OpenAI is using?

I'm thinking it would be better to just extract bos/eos from metadata instead of allowing user to set them from command line. I've been working on some improvements to `GGUFReader`...

Just checking, in case you had a particular use for swapping out BOS/EOS. A `--no-bos` could be useful I guess. This script can be really useful for generating prompts for...

This really belongs to [llama-cpp-python](https://github.com/abetlen/llama-cpp-python), but it looks like your issue is that you didn't install/compile the CUDA version, see the `Installation Configuration`->`CUDA` section in the [docs](https://llama-cpp-python.readthedocs.io/en/latest/).