langchain icon indicating copy to clipboard operation
langchain copied to clipboard

unknown command `MODULE`, with args beginning with: `LIST`,

Open denisa-ms opened this issue 1 year ago • 7 comments

System Info

when trying to connect to azure redis I get the following error: unknown command MODULE, with args beginning with: LIST, Here is the code:

fileName = "somefile.pdf" loader = PyPDFLoader(fileName) docs = loader.load_and_split() redis_conn = f"rediss://:{utils.REDIS_PWD}@{utils.REDIS_HOST}:{utils.REDIS_PORT}" rds = Redis.from_documents(docs, embeddings, redis_url=redis_conn, index_name='link')

Important: Connecting to redis like this, works! r = redis.StrictRedis(host=utils.REDIS_HOST,port=utils.REDIS_PORT, db=0, password=utils.REDIS_PWD, ssl=True) r.set('foo', 'bar') r.get('foo')

Who can help?

No response

Information

  • [ ] The official example notebooks/scripts
  • [X] My own modified scripts

Related Components

  • [ ] LLMs/Chat Models
  • [ ] Embedding Models
  • [ ] Prompts / Prompt Templates / Prompt Selectors
  • [ ] Output Parsers
  • [ ] Document Loaders
  • [X] Vector Stores / Retrievers
  • [ ] Memory
  • [ ] Agents / Agent Executors
  • [ ] Tools / Toolkits
  • [ ] Chains
  • [ ] Callbacks/Tracing
  • [ ] Async

Reproduction

This works: import redis r = redis.StrictRedis(host=utils.REDIS_HOST,port=utils.REDIS_PORT, db=0, password=utils.REDIS_PWD, ssl=True) r.set('foo', 'bar') r.get('foo')

This does not:

from langchain.document_loaders import PyPDFLoader # for loading the pdf from langchain.vectorstores.redis import Redis from langchain.embeddings import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model=OPENAI_EMBEDDING_MODEL_NAME, chunk_size=1) fileName = "somefile.pdf" loader = PyPDFLoader(fileName) docs = loader.load_and_split() redis_conn = f"rediss://:{utils.REDIS_PWD}@{utils.REDIS_HOST}:{utils.REDIS_PORT}" rds = Redis.from_documents(docs, embeddings, redis_url=redis_conn, index_name='link')

Expected behavior

save embeddings into REdis

denisa-ms avatar Jun 13 '23 03:06 denisa-ms

What's your redis client and server version? Server needs to be 7.0.0+ and redis python>=4.1.0 https://redis.io/commands/command-list/

chenyang-zheng avatar Jun 20 '23 08:06 chenyang-zheng

I am facing the same issue and found that Azure Redis has version 6.0.0 as the latest version. Thus unable to use the Azure Redis and langchain together. Is there a workaround for langchain to work with Redis server version 6.0.0 ?

Vishal240699 avatar Jun 26 '23 07:06 Vishal240699

Have the same issue with AWS Elasticache (Redis Cluster) running on engine v7.0 and redis==4.5.3

timxieICN avatar Jun 27 '23 19:06 timxieICN

For ElasticCache it's more like a restriction from AWS side that MODULE LIST command isn't supported and client.module_list() will fail. I've created my own vectorstore for Redis especially overwrite _check_redis_module_exist function to bypass it and which works fine. Also, ElasticCache DOES NOT support vector search as Redis Search, AWS Opensearch would be a better candidate for AWS ecosystem. https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/RestrictedCommands.html https://github.com/hwchase17/langchain/blob/a6b40b73e5f93e44df668272aaa672ea096db9ff/langchain/vectorstores/redis.py#L48

chenyang-zheng avatar Jun 29 '23 10:06 chenyang-zheng

@SherL0cked - can you give an example of how you overwrote the _check_redis_module_exist ? It looks like Redis Labs also restricts the MODULE LIST command

faynburd avatar Jul 09 '23 12:07 faynburd

@faynburd I have to create my own customized Redis VectorStore which is similar to existing langchain logic only changed the validations on https://github.com/hwchase17/langchain/blob/a6b40b73e5f93e44df668272aaa672ea096db9ff/langchain/vectorstores/redis.py#L48 https://github.com/hwchase17/langchain/blob/a6b40b73e5f93e44df668272aaa672ea096db9ff/langchain/vectorstores/redis.py#L139

chenyang-zheng avatar Jul 09 '23 15:07 chenyang-zheng

Actually managed with monkey patching this function to always return True

from langchain.vectorstores import redis

# command MODULE LIST is not supported by Redis Labs - monkey patch it
redis._check_redis_module_exist = lambda *args: True

faynburd avatar Jul 09 '23 20:07 faynburd

FYI, Redis.from_documents won't work on AWS Elasticache because there is no RediSearch installed on it, see here

DavidArenburg avatar Jul 25 '23 11:07 DavidArenburg

@faynburd, @DavidArenburg is right, AWS doesn't have, and will never have RediSearch support. Elasticache is a wrapper around OSS redis.

For the actual redis with redisearch, you can run in docker

❯ docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest

or start a free tier on redis cloud.

Spartee avatar Sep 12 '23 23:09 Spartee

Ok and for those for Microsoft

Azure Redis != Azure CAche for REdis Enterprise (ACRE)

and those from Google

MemoryStore != Redis Enteprise


Redis cloud supports all these clouds, theres marketplace offerings on all of them, and theres also just the docker I mentioned above.

Spartee avatar Sep 12 '23 23:09 Spartee

Hi, @denisa-ms,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, you reported an "unknown command MODULE" error when trying to connect to Azure Redis. There have been discussions and workarounds provided by other users, including customizing the Redis VectorStore and monkey patching the _check_redis_module_exist function. Additionally, there's discussion about the lack of RediSearch support on AWS Elasticache and alternative solutions such as running Redis with Redisearch in Docker or using Redis Cloud. The issue is currently unresolved.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

dosubot[bot] avatar Dec 13 '23 16:12 dosubot[bot]

Seems to be the problem when trying to use Google's Memorystore version of Redis. I was trying to run LangChain OpenGPTs (https://github.com/langchain-ai/opengpts) on Google Cloud Run using the integrated Redis feature of Run but I get this error:

redis.exceptions.ResponseError: unknown command 'MODULE', with args beginning with: 'LIST'

fovi-com avatar Jan 26 '24 03:01 fovi-com

Using RedisSemanticCache with an Azure Redis Service, getting same error. Anyone found the solution?

aboscus avatar Feb 14 '24 10:02 aboscus