Simon Willison

Maybe a migration like this: ```python @migration def m017_first_token_ms(db): db["responses"].add_column("first_token_ms", int) ``` Then record that value as `response.first_token_ms` at this point in the code, when the first chunk is yielded:...

I expanded that to: ```python @dataclass class Tool: name: str description: Optional[str] input_schema: BaseModel output_schema: BaseModel @classmethod def for_function(cls, function): """ Turn a Python function into a Tool object by:...

There's one edge-case where this could break: the `llm-anthropic` plugin has an option to output prefill text like this: https://github.com/simonw/llm-anthropic/blob/6c23b89c20dccf138c3e37d31853a796b5551106/llm_anthropic.py#L391-L403 ```python if stream: with messages_client.stream(**kwargs) as stream: if prefill_text: yield...

I'm torn on whether to use a Pydantic `BaseModel` for the internal `tool.input_schema` or if it would be more forward-thinking to use a JSON schema. The existing schemas feature supports...

Also worth noting that in `--no-stream` situations the time to first token measurement may not actually make sense. Maybe I should record `null` for those?

Does every LLM support fancy output schemas, or are there LLMs out there which can do tool calling but expect the answer to always be a string, not a JSON...

I'm a little nervous that there's some industry-standard way of calculating this that I'm unaware of. I'm going to take a risk and implement it with documentation as to how...

Now looks like this: https://github.com/simonw/llm/blob/86ccce4e5e37246e51b87eab27bb8503fa517900/llm/models.py#L109-L119 With clever code in `__post_init__()` to allow you to pass a Pydantic `BaseModel` to either `input_schema` or `output_schema` which gets converted into schema JSON for...

For plugins like `llm-mlx` it would be nice if the time spent loading the model from disk could be excluded from these calculations. Tricky to design that - would need...

I might change this later to make at least `input_schema` required - is it even a tool without it?