quarkus-langchain4j
quarkus-langchain4j copied to clipboard
Enable semantic cache with @CacheResult
In this PR, the idea is to implement the semantic cache as discussed in #637. The idea is to allow the developer to use the @CacheResult annotation to decide which methods need to use the semantic cache.
The idea is to create different classes following what was done for ChatMemory and use DefaultMemoryIdProvider to create an isolated cache for each user if possible (I don't know if this is enough to avoid possible data leakage).
An open point is what to do if the @Tools annotation is present.
Shouldn't @CacheResult be usable in this case?
TODO LIST:
- Improve tests
- Understand how to implement the cache concept for stream responses.
- Write documentation
There is still work to be done, but I think there is enough code to understand if the direction taken is the right one or if changes need to be made.