GPTCache
GPTCache copied to clipboard
[DOCS]: Add support for remote databases in GPTCache server config
Documentation Link
https://github.com/zilliztech/GPTCache/blob/main/examples/README.md#How-to-use-GPTCache-server
Describe the problem
My database service is running in another docker container, I tried to create a separate GPTCache server, but couldn't find the documentation. Here is the code without the GPTCache server:
DSN = f"postgresql+psycopg2://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}"
onnx = Onnx()
data_manager = get_data_manager(CacheBase(name='postgresql',sql_url=DSN), VectorBase("pgvector", dimension=onnx.dimension,url=DSN))
cache.init(
embedding_func=onnx.to_embeddings,
data_manager=data_manager,
similarity_evaluation=SearchDistanceEvaluation(),
)
cache.set_openai_key()
Could you please let me know how to move them in config.yaml to start a GPTCache server?
Anything else?
Here is a sample of config.yaml:
embedding:
onnx
embedding_config:
# Set embedding model params here
storage_config:
data_dir:
gptcache_data
manager:
postgresql,pgvector
vector_params:
# Set vector storage related params here
evaluation:
distance
evaluation_config:
# Set evaluation metric kws here
pre_function:
get_prompt
post_function:
first
config:
similarity_threshold: 0.8
# Set other config here
You can add the scalar parmas under storage config:
embedding:
onnx
embedding_config:
# Set embedding model params here
storage_config:
data_dir:
gptcache_data
manager:
postgresql,pgvector
vector_params:
# Set vector storage related params here
scalar_params:
sql_url: <URL>
table_name: <TABLE_NAME>
evaluation:
distance
evaluation_config:
# Set evaluation metric kws here
pre_function:
get_prompt
post_function:
first
config:
similarity_threshold: 0.8
# Set other config here