prompttools
prompttools copied to clipboard
hegelai
LlamaIndex Integration
🚀 The feature
We need an experiment or example for LlamaIndex. This will be similar to the work we need to do for LangChain and MindsDB.
Motivation, pitch
LlamaIndex is a popular framework for connecting LLMs to data. It would be good to support LlamaIndex testing using our existing vectorDB and LLM evaluations.
Alternatives
N/A
Additional context
N/A
hi @steventkrawczyk I'd love to work on this.
My guess is we'd need abstractions over LlamaIndex over core prompttools classes, but we'll need to update it with each release of LlamaIndex.
Hey @rachittshah we'd love your help! I think you're right that we'll need to build prompttools abstractions on top of the llamaindex abstractions. I'm hoping there are a few abstractions which are stable enough for us to build once, but if we need to update with each release we can explore that too.
Feel free to open a draft PR with what you're thinking now, and we can work together to iterate. If you want to chat in depth, I can add you to the dev channel of our discord, and we can set up a call to brainstorm if you'd find it useful.
Discord invite: https://discord.gg/72a9xh5Z7P
Hi @steventkrawczyk ! Joining discord, I think I'd open a draft PR with a proposed implementation of changing(since this would be a breaking change).
Joining discord, and keen to work on this!
+1 on this feature 😊 A client I work uses llama-index and there aren't notable libraries for LLM eval that are compatible. Yesterday deeplearning.ai released a course on RAG and LLM Eval using llama-index and truelens but I feel that promptools design would feel more natural and provide a simpler integration.
