getzep
Any good open source LLMs for zep's summarization?
This is not a feature/bug, just a question, since there is no discussion board I'll leave it here.
I have multiple hugging face models hosted and running OpenAI compatible endpoints. After hooking them up to Zep, they continue to work, however the results were kind of disappointing for the models I have tried. They get stuck in the whole Led Zeppelin part of the default prompt template, the intention extraction starts spinning up weird things and gets stuck up on Nike.
Has anyone used any huggingface LLMs for this with any degree of success? If so, please provide name and template you used if you changed the default template.
PS: It would be nice to be able to change intent prompt template in order to make it match the prompting style of selected model. (Someone has raised this already)
We're in the middle of a significant rewrite of Zep's LLM use, with far more flexibility around the use of different models. This will be released towards the end of the quarter.
We've made significant changes to Zep's operation in the last week. Facts are now Zep's primary memory artifact, and summaries are no longer generated similarly. You may use any model with an OpenAI-compatible API. However, we recommend using OpenAI, Anthropic, or larger Llama 3 models.