GPTCache

GPTCache copied to clipboard

Current Behavior

OpenAI prompt contains user role and system role message, as example below: message = [ {"role": "system", "content": " You are AI assistant to help answer question"}, {"role": "user", "content": "What is capital of USA?" } ]

But gptCache only consider and caches "user" role part. If two messages have same "user" msg but different "system" msg, they will be considered the same item, and the cached result will be used by both.

Expected Behavior

If two messages have same "user" msg but different "system" msg, they MUST not be considered the same cache item.

Steps To Reproduce

No response

Environment

Ubuntu 22.04 x64

Anything else?

No response

@qdaoming you can try to use the different pre_process_fun, like: all_content

@SimFG Thanks! Using this function works. I thought that it should be enabled by default.