Simon Willison

I think I'll treat audio/image responses separately from annotations - I'll use an expanded version of the existing **attachments** mechanism for that - including the existing `attachments` database table: https://github.com/simonw/llm/blob/656d8fa3c46f5babf8ee02e57fb0a4060e2da817/docs/logging.md?plain=1#L181-L194...

After [brainstorming with Claude](https://gist.github.com/simonw/2714309612f913e15368061ee0624aef) I think a solution to the terminal representation challenge could be to add markers around the annotated spans of text and then display those annotations below....

For DeepSeek reasoner that might look like this: ``` 「Okay, so I need to come up with a joke about a pelican and a walrus running a tea room together....

For the Python layer this might look like so: ```python response = llm.prompt("prompt goes here") print(response.text()) # outputs the plain text print(response.annotations) # Outputs annotations, see below for annotated in...

Then the SQL table design is pretty simple: ```sql CREATE TABLE [response_annotations] ( [id] INTEGER PRIMARY KEY, [response_id] TEXT REFERENCES [responses]([id]), [start_index] INTEGER, [end_index] INTEGER, [annotation] TEXT -- JSON );...

It bothers me very slightly that this design allows for exact positioning of annotations in a text stream response (with a start and end index) but doesn't support that for...

> asking the obvious question, why not use the academic paper style of using `[]` to reference instead of quoting the beginning text as anchor? I guess one reason is...

Or with numbers at the start: ``` Based on the document, here are the key trends in AI/LLMs from 2024: 1. Breaking the GPT-4 Barrier: [1]「The GPT-4 barrier was completely...

This came up again today thanks to Claude 3.7 Sonnet exposing thinking tokens: - https://github.com/simonw/llm-anthropic/issues/14 Based on that (and how it works in the Claude streaming API - see example...

Lots of these in the new OpenAI Responses API https://platform.openai.com/docs/api-reference/responses/create