llm
llm copied to clipboard
Enhancement Proposal: Track Head (for backtracking)
Sometimes it would be nice to step back a single generation. To fix a typo or redo a generation. This PR makes that possible. This is done by adding a parent ID to the responses and adding a table to track the HEAD ID.
I can clean it up and add some tests, but first I wanted to check if this was something you would be amenable to.
Here is a demonstration of what it allows for:
> llm 'Hello World. Reply with "hello".'
hello
> llm --continue 'Magic word is pineapple. Confirm with "okay".'
okay
> llm --continue 'What is the magic word?'
pineapple
> llm head back
Head moved back to response 01jctw7f1a2xv2n8ckjyhxr3k7
> llm head back
Head moved back to response 01jctw6rh9pd2kvphs0sdcjz8e
> llm --continue 'What is the magic word?'
please
> llm head set 01jctw7f1a2xv2n8ckjyhxr3k7
Head is now at response 01jctw7f1a2xv2n8ckjyhxr3k7
> llm --continue 'What is the magic word?'
pineapple
I think this is useful on its own and not too big of a change. However, this would also make it possible to explore many branches of a conversation in parallel, in a loom-like manner. I would make that as an additional module, but for that to be possible the responses need parent ids, so at least that much would have to be added to the db I believe.
Oh, I also changed the way 'llm --continue' works. Currently it tracks the conversation with the most recent start time. I found it much more helpful to track the conversation that has the most recent generations.
I have started turning this into a plugin, but I have to modify the migrations and do a lot of monkey-patching so its probably not something most people should use. But if you like you can close this.
https://github.com/FergusFettes/llm-head