dspy
dspy copied to clipboard
stanfordnlp
tree-of-thoughts-implementation
Hello! I think it would be very helpful for there to be a tutorial on how to add new primitive modules (e.g., Chain of Thought, Chain of Thought with Hint, etc.). It's not clear to me how I would add a method like Tree of Thoughts, for example.
If there's a rather straightforward way to do this and I've missed something, I'd be happy to throw together a tutorial notebook.
When implementing Tree Of Thought in the past, I've found that you can save a lot of compute if your language model supports Key/Value caching. That is, when you branch from a node, you typically give each child all the text input/output of its ancestors. But it's not efficient running the transformer model on all of those identical tokens again.
Obviously you can't do key/value caching with most API'ed models, like OpenAI's. But you can do with with hugging face's transformer library.
I implemented Tree of Thought in DSPy here: https://github.com/stanfordnlp/dspy/compare/main...thomasahle:dspy:main But it is not very good yet. Comments are very welcome.
It was merged into main? Can you please drop a link to PR.
It was merged into main? Can you please drop a link to PR.
