Simon Willison

I'd like to run an image model in `llama-cpp-python` - this one would be good: https://huggingface.co/openbmb/MiniCPM-V-2_6-gguf/tree/main The docs at https://llama-cpp-python.readthedocs.io/en/latest/#multi-modal-models seem to want a path to a CLIP model though,...

https://huggingface.co/xtuner/llava-llama-3-8b-v1_1-gguf would be a good one to figure out the Python / `llama-cpp-python` recipe for too.

Thank you! That’s exactly what I needed to know.

This research informed the attachments feature shipped in: - #590

Current schema: https://github.com/simonw/llm/blob/e867e13d1bb06d14fca5176e8bfe4dae2db000a3/docs/logging.md#L146-L168

Since some of these features are specific to Claude, it may make sense to have some kind of JSON column that specific LLM plugins can use to record information that...

Does it even make sense for a conversation that was "real" (user submits a prompt, gets a model response, sends a reply, gets another model response) to be stored in...

Worth considering tools here too, since those may require special thought about how to construct the schema.

I think "extras" might be a good way to model extra things that are specific to certain models. I could have a `extras_json` column for storing these. For places where...

Here's a schema consideration from tool use with Claude - a response might come back like this: https://docs.anthropic.com/en/docs/build-with-claude/tool-use#example-api-response-with-a-tool-use-content-block ```json { "id": "msg_01Aq9w938a90dw8q", "model": "claude-3-5-sonnet-20240620", "stop_reason": "tool_use", "role": "assistant", "content": [...