llm icon indicating copy to clipboard operation
llm copied to clipboard

Enhancement Proposal: Track Head (for backtracking)

Open FergusFettes opened this issue 1 year ago • 2 comments

Sometimes it would be nice to step back a single generation. To fix a typo or redo a generation. This PR makes that possible. This is done by adding a parent ID to the responses and adding a table to track the HEAD ID.

I can clean it up and add some tests, but first I wanted to check if this was something you would be amenable to.

Here is a demonstration of what it allows for:

> llm 'Hello World. Reply with "hello".'
hello
> llm --continue 'Magic word is pineapple. Confirm with "okay".'
okay
> llm --continue 'What is the magic word?'
pineapple
> llm head back
Head moved back to response 01jctw7f1a2xv2n8ckjyhxr3k7
> llm head back
Head moved back to response 01jctw6rh9pd2kvphs0sdcjz8e
> llm --continue 'What is the magic word?'
please
> llm head set 01jctw7f1a2xv2n8ckjyhxr3k7
Head is now at response 01jctw7f1a2xv2n8ckjyhxr3k7
> llm --continue 'What is the magic word?'
pineapple

I think this is useful on its own and not too big of a change. However, this would also make it possible to explore many branches of a conversation in parallel, in a loom-like manner. I would make that as an additional module, but for that to be possible the responses need parent ids, so at least that much would have to be added to the db I believe.

FergusFettes avatar Nov 16 '24 16:11 FergusFettes

Oh, I also changed the way 'llm --continue' works. Currently it tracks the conversation with the most recent start time. I found it much more helpful to track the conversation that has the most recent generations.

FergusFettes avatar Nov 16 '24 16:11 FergusFettes

I have started turning this into a plugin, but I have to modify the migrations and do a lot of monkey-patching so its probably not something most people should use. But if you like you can close this.

https://github.com/FergusFettes/llm-head

FergusFettes avatar Feb 10 '25 07:02 FergusFettes