Trouble with QdrantRetrieveUserProxyAgent in a Docker-Hosted Qdrant Vector DB

[libin85]

Describe the issue

Issue Description I am experiencing difficulties using QdrantRetrieveUserProxyAgent to retrieve embeddings based on user queries. Despite following the standard configuration and setup, I keep encountering an error when attempting to retrieve data.

Current Implementation

import autogen
from autogen.agentchat.contrib.retrieve_assistant_agent import RetrieveAssistantAgent
from autogen.agentchat.contrib.qdrant_retrieve_user_proxy_agent import QdrantRetrieveUserProxyAgent
from qdrant_client import QdrantClient

# Configuration
config_list = autogen.config_list_from_json(env_or_file="OAI_CONFIG_LIST")

# Assistant Agent
assistant = RetrieveAssistantAgent(
    name="assistant",
    system_message="You are a helpful assistant.",
    llm_config={"request_timeout": 600, "seed": 42, "config_list": config_list}
)

# Qdrant Client
client = QdrantClient(url="https://qdranturl", port=443, api_key="1234")

# Qdrant Proxy Agent
ragproxyagent = QdrantRetrieveUserProxyAgent(
    name="qdrantagent",
    human_input_mode="NEVER",
    is_termination_msg=None,
    max_consecutive_auto_reply=5,
    retrieve_config={"task": "default", "client": client, "collection_name": "lj-dev-collection"}
)

def agent_call(user_query: str):
    assistant.reset()
    ragproxyagent.initiate_chat(assistant, problem=user_query)

Error Encountered When executing the agent_call function with a user query, the following error is returned: File "C:\testproject\myenv\Lib\site-packages\qdrant_client\http\api_client.py", line 97, in send raise UnexpectedResponse.for_response(response) qdrant_client.http.exceptions.UnexpectedResponse: Unexpected Response: 400 (Bad Request) Raw response content: b'{"status":{"error":"Wrong input: Vector params for fast-bge-small-en are not specified in config"},"time":0.000031489}'

Attempts to Resolve I have tried different versions of qdrant-client, including qdrant-client[fastembed]. The embeddings were created using the text-embedding-ada-002 model.

Request for Assistance I would appreciate any insights or suggestions on how to resolve this issue, particularly regarding the configuration of the QdrantRetrieveUserProxyAgent or any missing parameters in the Qdrant client setup.

Steps to reproduce

No response

Screenshots and logs

No response

Additional Information

Environment Hosting: Docker Compose Database: Qdrant Vector DB Language: Python Libraries: autogen, qdrant_client

PIP Packages pyautogen==0.2.6 pyautogen[retrievechat]==0.2.6 qdrant-client==1.7.0

[libin85]

@Anush008 : Appreciate if you share your thoughts on this

[Anush008]

@libin85, I'll take a look.

[libin85]

Thank you Anush

[Anush008]

Hey @libin85, from your error message, it is clear that named vectors are causing the issue.

QdrantRetrieveUserProxyAgent uses Qdrant's FastEmbed library to generate and query embeddings. The FastEmbed integration creates named vectors for the points. For instance, if the chosen model is BAAI/bge-small-en (Default), a named vector 'fast-bge-small-en' is configured for the collection by the Qdrant client.

Since you're using a different model for the embeddings and your collection lacks this named vector, you're running into this error.

To resolve this, you can either use QdrantRetrieveUserProxyAgent for document upload or alternatively, utilize the Qdrant client's add() method, which generates embeddings and adds them as appropriately named vectors to the collection.

[libin85]

@Anush008 So if I understand you correctly. We can only use QdrantRetrieveUserProxyAgent for retrieval only if we use the same agent for creating embeddings. If the embeddings are created using another service it might not work. Am i correct with my assumptions?

[Anush008]

Yes.

[dsalas-crogl]

HI @libin85 , I was just curious if you ever got the QdrantRetrieveUserProxyAgent module working? I ran into the same issue that you highlighted above. I was just curious if you ended up doing something different. Using the examples from https://microsoft.github.io/autogen/blog/2023/10/18/RetrieveChat#read-more, I'm playing around with wrapping RetrieveUserProxyAgent. That does look a little more promising, but not sure if it's the best the approach.