langchain
langchain copied to clipboard
langchain-ai
add sqlalchemy generic cache
Created a generic SQLAlchemyCache class to plug any database supported by SQAlchemy. (I am using Postgres). I also based the class SQLiteCache class on this class SQLAlchemyCache.
As a side note, I'm questioning the need for two distinct class LLMCache, FullLLMCache. Shouldn't we merge both ?
The fact that llm is a text field is quite not optimal for Postgres (who support JSONB). What do you think?
As a side note, I'm questioning the need for two distinct class LLMCache, FullLLMCache. Shouldn't we merge both ?
yeah - this is due to some ugliness in the base class and differences between generate and call - i have an idea though, let me put something up shortly ill tag you
The fact that llm is a text field is quite not optimal for Postgres (who support JSONB). What do you think?
trying to understand - you'd prefer to save the llm as jsonb why? so you can load it later?
@hwchase17 I haven't thought about loading it later, but it could be useful 🤔
Anyway, I like to query my cache for analysis. Different use cases:
- displaying/filtering model parameters -> It's better to have the model parameters as jsonb.
- check last call prompt / result -> It would be great to have an "id" call sequence field.
Lastly, it's useful to be able to join cache rows with other information / table.
In one of my projet, I'm doing 4-5 calls per user query. I have added "queryId" as one of the field of the cache table. Really useful to debug things.