langchain
langchain copied to clipboard
langchain-ai
vectorstore redis return TypeError
System Info
Langchain version: 0.0.173 Redis version: 4.5.5 Python version: 3.11 OS: Windows 10
Who can help?
@hwchase17 @agola11
Information
- [X] The official example notebooks/scripts
- [ ] My own modified scripts
Related Components
- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async
Reproduction
using latest langchain with redis vectorstore return error TypeError code here:
embeddings = OpenAIEmbeddings(model="text-embedding-ada-002", chunk_size=1)
rds = Redis.from_texts(texts, embeddings, redis_url="redis://localhost:6379", index_name=uid)
error stack:
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
File D:\Projects\openai\venv\Lib\site-packages\redis\connection.py:1450, in ConnectionPool.get_connection(self, command_name, *keys, **options)
1449 try:
-> 1450 connection = self._available_connections.pop()
1451 except IndexError:
IndexError: pop from empty list
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
Cell In[5], line 11
8 print("create vector store with id: {}".format(uid))
10 embeddings = OpenAIEmbeddings(model="text-embedding-ada-002", chunk_size=1)
---> 11 rds = Redis.from_texts(texts, embeddings, redis_url = "redis://localhost:6379",index_name=uid)
File D:\Projects\openai\venv\Lib\site-packages\langchain\vectorstores\redis.py:448, in Redis.from_texts(cls, texts, embedding, metadatas, index_name, content_key, metadata_key, vector_key, **kwargs)
419 @classmethod
420 def from_texts(
421 cls: Type[Redis],
(...)
429 **kwargs: Any,
430 ) -> Redis:
431 """Create a Redis vectorstore from raw documents.
432 This is a user-friendly interface that:
433 1. Embeds documents.
(...)
446 )
447 """
--> 448 instance, _ = cls.from_texts_return_keys(
449 texts=texts,
450 embedding=embedding,
451 metadatas=metadatas,
452 index_name=index_name,
453 content_key=content_key,
454 metadata_key=metadata_key,
455 vector_key=vector_key,
456 kwargs=kwargs,
457 )
458 return instance
File D:\Projects\openai\venv\Lib\site-packages\langchain\vectorstores\redis.py:399, in Redis.from_texts_return_keys(cls, texts, embedding, metadatas, index_name, content_key, metadata_key, vector_key, distance_metric, **kwargs)
396 index_name = uuid.uuid4().hex
398 # Create instance
--> 399 instance = cls(
400 redis_url=redis_url,
401 index_name=index_name,
402 embedding_function=embedding.embed_query,
403 content_key=content_key,
404 metadata_key=metadata_key,
405 vector_key=vector_key,
406 **kwargs,
407 )
409 # Create embeddings over documents
410 embeddings = embedding.embed_documents(texts)
File D:\Projects\openai\venv\Lib\site-packages\langchain\vectorstores\redis.py:138, in Redis.__init__(self, redis_url, index_name, embedding_function, content_key, metadata_key, vector_key, relevance_score_fn, **kwargs)
136 redis_client = redis.from_url(redis_url, **kwargs)
137 # check if redis has redisearch module installed
--> 138 _check_redis_module_exist(redis_client, REDIS_REQUIRED_MODULES)
139 except ValueError as e:
140 raise ValueError(f"Redis failed to connect: {e}")
File D:\Projects\openai\venv\Lib\site-packages\langchain\vectorstores\redis.py:48, in _check_redis_module_exist(client, required_modules)
46 def _check_redis_module_exist(client: RedisType, required_modules: List[dict]) -> None:
47 """Check if the correct Redis modules are installed."""
---> 48 installed_modules = client.module_list()
49 installed_modules = {
50 module[b"name"].decode("utf-8"): module for module in installed_modules
51 }
52 for module in required_modules:
File D:\Projects\openai\venv\Lib\site-packages\redis\commands\core.py:5781, in ModuleCommands.module_list(self)
5774 def module_list(self) -> ResponseT:
5775 """
5776 Returns a list of dictionaries containing the name and version of
5777 all loaded modules.
5778
5779 For more information see https://redis.io/commands/module-list
5780 """
-> 5781 return self.execute_command("MODULE LIST")
File D:\Projects\openai\venv\Lib\site-packages\redis\client.py:1266, in Redis.execute_command(self, *args, **options)
1264 pool = self.connection_pool
1265 command_name = args[0]
-> 1266 conn = self.connection or pool.get_connection(command_name, **options)
1268 try:
1269 return conn.retry.call_with_retry(
1270 lambda: self._send_command_parse_response(
1271 conn, command_name, *args, **options
1272 ),
1273 lambda error: self._disconnect_raise(conn, error),
1274 )
File D:\Projects\openai\venv\Lib\site-packages\redis\connection.py:1452, in ConnectionPool.get_connection(self, command_name, *keys, **options)
1450 connection = self._available_connections.pop()
1451 except IndexError:
-> 1452 connection = self.make_connection()
1453 self._in_use_connections.add(connection)
1455 try:
1456 # ensure this connection is connected to Redis
File D:\Projects\openai\venv\Lib\site-packages\redis\connection.py:1492, in ConnectionPool.make_connection(self)
1490 raise ConnectionError("Too many connections")
1491 self._created_connections += 1
-> 1492 return self.connection_class(**self.connection_kwargs)
File D:\Projects\openai\venv\Lib\site-packages\redis\connection.py:956, in Connection.__init__(self, host, port, socket_timeout, socket_connect_timeout, socket_keepalive, socket_keepalive_options, socket_type, **kwargs)
954 self.socket_keepalive_options = socket_keepalive_options or {}
955 self.socket_type = socket_type
--> 956 super().__init__(**kwargs)
TypeError: AbstractConnection.__init__() got an unexpected keyword argument 'kwargs'
Expected behavior
I followed the instructions in the documentation and it ran fine before, but when I upgraded the langchain version to the latest version, it failed. I believe this is related to https://github.com/hwchase17/langchain/pull/4857, am I using this method incorrectly now? I'm new to langchain.
believe this is related to #4936 (should make it into next release 0.0.174)
believe this is related to #4936 (should make it into next release 0.0.174)
Thanks for such quick response! It works in v0.0.168, so i move back for usage, and looking forward to next release
I'm running langchain version 0.0.177 and am still seeing the same error. was it supposed to be fixed?
This is it: Redis.from_texts(documents, embeddings, redis_url="redis://localhost:6379", index_name='link') Traceback (most recent call last):
Cell In[12], line 1 Redis.from_texts(documents, embeddings, redis_url="redis://localhost:6379", index_name='link')
File ~\anaconda3\envs\ionix\Lib\site-packages\langchain\vectorstores\redis.py:449 in from_texts instance, _ = cls.from_texts_return_keys(
File ~\anaconda3\envs\ionix\Lib\site-packages\langchain\vectorstores\redis.py:400 in from_texts_return_keys instance = cls(
File ~\anaconda3\envs\ionix\Lib\site-packages\langchain\vectorstores\redis.py:139 in init _check_redis_module_exist(redis_client, REDIS_REQUIRED_MODULES)
File ~\anaconda3\envs\ionix\Lib\site-packages\langchain\vectorstores\redis.py:48 in _check_redis_module_exist installed_modules = client.module_list()
ive encountered this as well and while its not a pr or permanent code fix heres a temporary solution you can add to your code
if isinstance(texts, str): texts = [texts] #print(f"{len(texts)}") elif isinstance(texts, list): texts = [str(text) for text in texts] #print(f"{texts[1]}") else: raise TypeError("texts must be a string or a list of strings")
I'm using it inline just before my redis,from_documents call
Thanks, but I'm using a loader to create the texts. it creates a list of langchain.schema.Document objects to apply the embeddings on. are you saying that converting these to strings would solve the problem? can you even use strings with from_documents()? I'm getting the following which suggests that using from_documents requires langchain.schema.Document objects.
**flatdoc=[str(x) for x in documents]
v2=Redis.from_documents(flatdoc, embeddings, redis_url="redis://localhost:6379", index_name='link') Traceback (most recent call last):
Cell In[12], line 1 v2=Redis.from_documents(flatdoc, embeddings, redis_url="redis://localhost:6379", index_name='link')
File ~\anaconda3\envs\ionix\Lib\site-packages\langchain\vectorstores\base.py:305 in from_documents texts = [d.page_content for d in documents]
File ~\anaconda3\envs\ionix\Lib\site-packages\langchain\vectorstores\base.py:305 in
AttributeError: 'str' object has no attribute 'page_content**'
regardless, if I flatten to strings and use from_texts() to try and create and load the embeddings I'm getting the old error: v2=Redis.from_texts(flatdoc, embeddings, redis_url="redis://localhost:6379", index_name='link') Traceback (most recent call last):
**Cell In[14], line 1 v2=Redis.from_texts(flatdoc, embeddings, redis_url="redis://localhost:6379", index_name='link')
File ~\anaconda3\envs\ionix\Lib\site-packages\langchain\vectorstores\redis.py:449 in from_texts instance, _ = cls.from_texts_return_keys(
File ~\anaconda3\envs\ionix\Lib\site-packages\langchain\vectorstores\redis.py:400 in from_texts_return_keys instance = cls(
File ~\anaconda3\envs\ionix\Lib\site-packages\langchain\vectorstores\redis.py:139 in init _check_redis_module_exist(redis_client, REDIS_REQUIRED_MODULES)
File ~\anaconda3\envs\ionix\Lib\site-packages\langchain\vectorstores\redis.py:48 in _check_redis_module_exist installed_modules = client.module_list()
AttributeError: 'Redis' object has no attribute 'module_list'**
if documents contains Document objects not strings then you should use the Redis.from_documents method, not from_texts
I'm loading both documents and text and then using either Redis.from_existing_index, Redis.from_texts, Redis.from_documents, and as a retriever class using retriever.get_relevant_documents all with success. No i'm not saying that it is expecting a string, it seems that it is expecting a list of strings. but my background is also R not python so i could be mistaken.
also this is how my create instance looks in redis.py
# Create instance instance = cls( redis_url=redis_url, index_name=index_name, embedding_function=embedding.embed_query, content_key=content_key, metadata_key=metadata_key, vector_key=vector_key, **kwargs,
and
instance, _ = cls.from_texts_return_keys( texts=texts, embedding=embedding, metadatas=metadatas, index_name=index_name, content_key=content_key, metadata_key=metadata_key, vector_key=vector_key, kwargs=kwargs, **kwargs, )
also throws ResponseError: link: no such index ,
251 return [doc for doc, _ in docs_and_scores]
File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/langchain/vectorstores/redis.py:327](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/langchain/vectorstores/redis.py:327), in Redis.similarity_search_with_score(self, query, k)
320 params_dict: Mapping[str, str] = {
321 "vector": np.array(embedding) # type: ignore
322 .astype(dtype=np.float32)
323 .tobytes()
324 }
326 # Perform vector search
--> 327 results = self.client.ft(self.index_name).search(redis_query, params_dict)
329 # Prepare document results
330 docs = [
331 (
332 Document(
(...)
337 for result in results.docs
338 ]
File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/commands/search/commands.py:420](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/commands/search/commands.py:420), in SearchCommands.search(self, query, query_params)
418 args, query = self._mk_query_args(query, query_params=query_params)
419 st = time.time()
--> 420 res = self.execute_command(SEARCH_CMD, *args)
422 if isinstance(res, Pipeline):
423 return res
File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1269](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1269), in Redis.execute_command(self, *args, **options)
1266 conn = self.connection or pool.get_connection(command_name, **options)
1268 try:
-> 1269 return conn.retry.call_with_retry(
1270 lambda: self._send_command_parse_response(
1271 conn, command_name, *args, **options
1272 ),
1273 lambda error: self._disconnect_raise(conn, error),
1274 )
1275 finally:
1276 if not self.connection:
File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/retry.py:46](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/retry.py:46), in Retry.call_with_retry(self, do, fail)
44 while True:
45 try:
---> 46 return do()
47 except self._supported_errors as error:
48 failures += 1
File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1270](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1270), in Redis.execute_command..()
1266 conn = self.connection or pool.get_connection(command_name, **options)
1268 try:
1269 return conn.retry.call_with_retry(
-> 1270 lambda: self._send_command_parse_response(
1271 conn, command_name, *args, **options
1272 ),
1273 lambda error: self._disconnect_raise(conn, error),
1274 )
1275 finally:
1276 if not self.connection:
File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1246](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1246), in Redis._send_command_parse_response(self, conn, command_name, *args, **options)
1242 """
1243 Send a command and parse the response
1244 """
1245 conn.send_command(*args)
-> 1246 return self.parse_response(conn, command_name, **options)
File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1286](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1286), in Redis.parse_response(self, connection, command_name, **options)
1284 options.pop(NEVER_DECODE)
1285 else:
-> 1286 response = connection.read_response()
1287 except ResponseError:
1288 if EMPTY_RESPONSE in options:
File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/connection.py:897](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/connection.py:897), in AbstractConnection.read_response(self, disable_decoding, disconnect_on_error)
894 self.next_health_check = time() + self.health_check_interval
896 if isinstance(response, ResponseError):
--> 897 raise response
898 return response
ResponseError: link: no such index
also throws
ResponseError: link: no such index,251 return [doc for doc, _ in docs_and_scores] File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/langchain/vectorstores/redis.py:327](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/langchain/vectorstores/redis.py:327), in Redis.similarity_search_with_score(self, query, k) 320 params_dict: Mapping[str, str] = { 321 "vector": np.array(embedding) # type: ignore 322 .astype(dtype=np.float32) 323 .tobytes() 324 } 326 # Perform vector search --> 327 results = self.client.ft(self.index_name).search(redis_query, params_dict) 329 # Prepare document results 330 docs = [ 331 ( 332 Document( (...) 337 for result in results.docs 338 ] File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/commands/search/commands.py:420](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/commands/search/commands.py:420), in SearchCommands.search(self, query, query_params) 418 args, query = self._mk_query_args(query, query_params=query_params) 419 st = time.time() --> 420 res = self.execute_command(SEARCH_CMD, *args) 422 if isinstance(res, Pipeline): 423 return res File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1269](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1269), in Redis.execute_command(self, *args, **options) 1266 conn = self.connection or pool.get_connection(command_name, **options) 1268 try: -> 1269 return conn.retry.call_with_retry( 1270 lambda: self._send_command_parse_response( 1271 conn, command_name, *args, **options 1272 ), 1273 lambda error: self._disconnect_raise(conn, error), 1274 ) 1275 finally: 1276 if not self.connection: File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/retry.py:46](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/retry.py:46), in Retry.call_with_retry(self, do, fail) 44 while True: 45 try: ---> 46 return do() 47 except self._supported_errors as error: 48 failures += 1 File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1270](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1270), in Redis.execute_command..() 1266 conn = self.connection or pool.get_connection(command_name, **options) 1268 try: 1269 return conn.retry.call_with_retry( -> 1270 lambda: self._send_command_parse_response( 1271 conn, command_name, *args, **options 1272 ), 1273 lambda error: self._disconnect_raise(conn, error), 1274 ) 1275 finally: 1276 if not self.connection: File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1246](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1246), in Redis._send_command_parse_response(self, conn, command_name, *args, **options) 1242 """ 1243 Send a command and parse the response 1244 """ 1245 conn.send_command(*args) -> 1246 return self.parse_response(conn, command_name, **options) File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1286](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/client.py:1286), in Redis.parse_response(self, connection, command_name, **options) 1284 options.pop(NEVER_DECODE) 1285 else: -> 1286 response = connection.read_response() 1287 except ResponseError: 1288 if EMPTY_RESPONSE in options: File [~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/connection.py:897](https://file+.vscode-resource.vscode-cdn.net/home/jimmycliff/Projects/Omic/GPT/langchain/docs/modules/indexes/vectorstores/examples/~/miniconda3/envs/biogpt/lib/python3.10/site-packages/redis/connection.py:897), in AbstractConnection.read_response(self, disable_decoding, disconnect_on_error) 894 self.next_health_check = time() + self.health_check_interval 896 if isinstance(response, ResponseError): --> 897 raise response 898 return response ResponseError: link: no such index
DId you change the link variable or use the one in the example?
i'm still seeing the issue. at bottom it's line 48 in redis.py: installed_modules = client.module_list() where the client is created in line 137: redis_client = redis.from_url(redis_url, **kwargs) # check if redis has redisearch module installed _check_redis_module_exist(redis_client, REDIS_REQUIRED_MODULES) but redis.client.Redis has no "module_list()" method (and neither does redis itself nor redis.client) What am i missing?
at version 0.0.183, no such Error happened, same code as i show above. so i think my issue here has been sovled. could be another problem?
This should be fixed, but also, some of you are likely not running a new enough version of Redis-Stack
❯ docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest
Will solve most of these
Hi, @axiangcoding,
I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, you encountered a TypeError when using the latest LangChain version with Redis vectorstore, but this issue has been resolved in version 0.0.183. There were discussions about potential fixes, and Spartee suggested that updating the Redis-Stack version could also resolve the issue for some users.
Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.
Thank you for your understanding and contributions to LangChain! If you have any further questions or need assistance, feel free to reach out.
![dosubot[bot] avatar](https://avatars.githubusercontent.com/in/324583?v=4 "Published by a pub.dev verified publisher"){kind=small}