chroma
chroma copied to clipboard
chroma-core
[Install issue]: install chroma on AWS lambda layer, exceeded size limit
What happened?
I have deployed the chroma docker compose image on my server. Now I would like to launch the chroma client library using an AWS lambda function. But the pip install chromadb creates a dependency zip around 220 MB which is too big for lambda layer. What is the most practical way of using chromadb on cloud functions like AWS Lambda?
Versions
python 3.8/3.9, AWS Lambda with layers
Relevant log output
No response
@PeterTF656 if I understand you correctly - you want to use a lambda to read and write to the persisted DB?
just a heads up that if you have 2 lambdas running at the same time, you will probably have events where you lose data
- Process1 reads
DB - Process2 reads
DB - Process1 writes
DB2with new info [1,2,3] - Process2 writes
DB2with new info [4,5,6]
[1,2,3] Will get stomped.
@PeterTF656 if I understand you correctly - you want to use a lambda to read and write to the persisted DB?
just a heads up that if you have 2 lambdas running at the same time, you will probably have events where you lose data
- Process1 reads
DB- Process2 reads
DB- Process1 writes
DB2with new info [1,2,3]- Process2 writes
DB2with new info [4,5,6][1,2,3] Will get stomped.
I have a server that is running the ChromaDB Docker image, and I was trying to figure out how to read and write to it using AWS Lambda. I'm more familiar with the AWS Apigateway + Lambda + DynamoDB workflow, so I initially thought of using a cloud function to perform CRUD operations on the database. Now it seems that a better solution might be to use the Chromadb client on the hosted server itself to handle these operations.
@PeterTF656 I see now. You could certainly use the python or JS client inside a lambda to talk to the backend hosted service. That would work fine.
@PeterTF656 I see now. You could certainly use the python or JS client inside a lambda to talk to the backend hosted service. That would work fine.
Thanks for your response. In the case of using a lambda function and lambda layers, the pip install chromadb command creates a 220 MB dependency zip and exceeds the size limit of the lambda layer... So I wonder if there's another way that can install chromadb on lambda and import the it to use the client.
@PeterTF656 Faced the same issue. I instead created a layer ignoring all chroma dependencies using --no-deps and installed just the ones my app needed. That way it does not package huge dependencies like torch. There is also an alternate way with the serverless framework where you can zip the requirements into the lambda itself (without unzipping so it does not exceed 250 mb). Ref - https://www.serverless.com/plugins/serverless-python-requirements (See "Dealing with Lambda's size limitations")
Also, the embedded DuckDB without persistence tries to create .chroma/index. This created on lambda because write is only allowed in /tmp. So as a workaround, you'll also need to persist it to /tmp. Ref issue - #477
Thank you! @strangest-quark In you case, did you also upload a pandas layer as well? I am using langchain + chromadb (with no deps), but it shows that I also need the pandas layer. But if I add the pandas layer, it goes above the size limit...
Not sure if anyone found a solution to this. You can import chroma using the chromadb-client package as a layer. But to query, you need the embeddings which blows up the zip file. Wondering if you were able to find any solution to this 2 weeks later... stuck on the same thing
@PeterTF656 we are going to remove pandas, - working on the PR right now
Not sure if anyone found a solution to this. You can import chroma using the chromadb-client package as a layer. But to query, you need the embeddings which blows up the zip file. Wondering if you were able to find any solution to this 2 weeks later... stuck on the same thing
Hi! I am using lambda image container (has a size limit of 10GB) now. Took me a while but overall it was manageable.
Hi! I am using lambda image container (has a size limit of 10GB) now. Took me a while but overall it was manageable.
How did you do that? I am trying to install chromadb but getting following error:
ERROR: Could not build wheels for chroma-hnswlib, which is required to install pyproject.toml-based projects
This is my Dockerfile:
FROM public.ecr.aws/lambda/python:3.10
# Copy requirements.txt
COPY requirements.txt ${LAMBDA_TASK_ROOT}
ENV HNSWLIB_NO_NATIVE=1
RUN echo "export HNSWLIB_NO_NATIVE=1" >> /etc/bash.bashrc
# Copy function code
COPY . ${LAMBDA_TASK_ROOT}/
# RUN yum install gcc-c++ -y
# Install the specified packages
RUN pip install -r requirements.txt
# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "lambda_function.lambda_handler" ]
@PeterTF656 Faced the same issue. I instead created a layer ignoring all chroma dependencies using
--no-depsand installed just the ones my app needed. That way it does not package huge dependencies like torch.
how did you manage to install required dependencies with --no-deps? it is telling one by one library which needs to be installed. can you provide a list of dependencies?
Hi, I have the same issue, is there any solution? I have Docker running on an EC2 server, and that's the one being fed with the data I need. With Lambda, all I have to do is query that data on that server. @PeterTF656
