llm icon indicating copy to clipboard operation
llm copied to clipboard

Published 20 hours ago • verified publisher icon simonw

Support prefill

Open simonw opened this issue 2 months ago • 8 comments

Claude 3 and other models (like Reka) support prefill, where you can construct a chat but set the first tokens of the model's reply. I use that in datasette-query-assistant here: https://github.com/datasette/datasette-query-assistant/blob/a777a80bcb3b42933b2933de895f4f2eb9376e9d/datasette_query_assistant/init.py#L52-L62

LLM should offer this at both the CLI level and the Python API level.

simonw avatar Apr 18 '24 23:04 simonw