Simon Willison

Handle errors from incorrect tool usage better - llm chat crashes if a tool name is invalid

9

This happened with Qwen 3 4B (Ollama): ``` llm chat -T QuickJS -m qwen3:4b --td Chatting with qwen3:4b Type 'exit' or 'quit' to exit Type '!multi' to enter multiple lines,...

llm -c should re-add Toolbox tools with serialized configuration

2

Spotted working on https://github.com/simonw/llm-tools-datasette ```bash llm -T 'Datasette("https://datasette.io/content")' 'show tables' --td llm -c 'Do something neat with the views' # Does not work ``` This relates to: - #1080 Need...

Files tool (toolbox) plugin

19

Out of the box `llm` should have at least one tool. I think a `Files` tool that can provide read/write access limited to a particular directory is a good idea....

New design for logs involving tool calls (and maybe tool classes)

8

> I am going to design the log output next, since that can inform the database schema. _Originally posted by @simonw in [#1059](https://github.com/simonw/llm/issues/1059#issuecomment-2907086013)_

> I had an idea to support multiple instances of the same tool - eg two different filesystem tools - using a suffix. I should revisit that idea. _Originally posted...

Monotonic ULIDs to fix the tests

4

I keep running into tests that fail in weird ways because ULIDs in the same ms don't sort reliably. It turns out the ULID spec covers this: https://github.com/ulid/spec/blob/master/README.md#monotonicity But the...

Full documentation for tools

4

Lots of stuff to document! New `tools.md` top level page. Preview: https://github.com/simonw/llm/blob/tools/docs/tools.md TODO: - [x] Document `register_tools()` plugin hook - [x] Basic explanation of what tools are and how they...

It would be convenient if you could set the `before_call` and `after_call` (and maybe even `chain_limit` too) when creating a `Conversation` that uses tools.

Embeddings need to handle truncation automatically (maybe via extra options)

3

I frustrating thing about embeddings right now is this: ```bash cd llm llm embed-multi --files docs/ '*.md' -d /tmp/docs.db docs ``` This fails with a messy error: ``` ... File...

Use condense_json() to make response logging more efficient

1

Most database log entries currently contain both the prompt and the response twice: ```bash llm logs -c --json ``` ```json [ { "id": "01jmddkan06b0568gjk90k134t", "model": "gpt-4o-mini", "prompt": "say hi", "system":...