undocumented sqlite3 dependency

[nick-youngblut]

Dockerfile:

# Use an official Python runtime as a base image
FROM python:3.11-slim-buster

# Set the working directory in the container
WORKDIR /crewai

# Install system dependencies
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
      gcc curl gnupg2 less vim \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Copy the requirements.txt file into the container
COPY requirements.txt ./

# Copy the package files into the container
#COPY ./crewai ./

# Install any needed Python packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

requirements.txt file:

openai==1.23.6
crewai==0.28.8
crewai-tools==0.1.7

Import:

python -c "import crewai"

Error:

ERROR: Could not find a version that satisfies the requirement pysqlite3-binary (from versions: none)
ERROR: No matching distribution found for pysqlite3-binary
2024-04-29 13:05:52,755 [embedchain] [ERROR] Failed to swap std-lib sqlite3 with pysqlite3 for ChromaDb compatibility. Error: Command '['/usr/local/bin/python', '-m', 'pip', 'install', 'pysqlite3-binary', '--quiet', '--disable-pip-version-check']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.11/site-packages/crewai/__init__.py", line 1, in <module>
    from crewai.agent import Agent
  File "/usr/local/lib/python3.11/site-packages/crewai/agent.py", line 23, in <module>
    from crewai.agents import CacheHandler, CrewAgentExecutor, CrewAgentParser, ToolsHandler
  File "/usr/local/lib/python3.11/site-packages/crewai/agents/__init__.py", line 2, in <module>
    from .executor import CrewAgentExecutor
  File "/usr/local/lib/python3.11/site-packages/crewai/agents/executor.py", line 16, in <module>
    from crewai.memory.entity.entity_memory_item import EntityMemoryItem
  File "/usr/local/lib/python3.11/site-packages/crewai/memory/__init__.py", line 1, in <module>
    from .entity.entity_memory import EntityMemory
  File "/usr/local/lib/python3.11/site-packages/crewai/memory/entity/entity_memory.py", line 3, in <module>
    from crewai.memory.storage.rag_storage import RAGStorage
  File "/usr/local/lib/python3.11/site-packages/crewai/memory/storage/rag_storage.py", line 7, in <module>
    from embedchain import App
  File "/usr/local/lib/python3.11/site-packages/embedchain/__init__.py", line 5, in <module>
    from embedchain.app import App  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/embedchain/app.py", line 33, in <module>
    from embedchain.vectordb.chroma import ChromaDB
  File "/usr/local/lib/python3.11/site-packages/embedchain/vectordb/chroma.py", line 4, in <module>
    from chromadb import Collection, QueryResult
  File "/usr/local/lib/python3.11/site-packages/chromadb/__init__.py", line 79, in <module>
    raise RuntimeError(
RuntimeError: Your system has an unsupported version of sqlite3. Chroma                     requires sqlite3 >= 3.35.0.
Please visit                     https://docs.trychroma.com/troubleshooting#sqlite to learn how                     to upgrade.

The installation docs do not state that sqlite3 must be installed, in addition to running pip install crewai.

[nick-youngblut]

At least with my Dockerfile setup, apt-get update && apt-get install sqlite3 will only install sqlite3 version 3.27.2 2019-02-25, but Chroma requires sqlite3 >= 3.35.0. The Chroma docs on this issue are non-trivial (e.g., https://gist.github.com/defulmere/8b9695e415a44271061cc8e272f3c300 or https://github.com/coleifer/pysqlite3?tab=readme-ov-file#building-a-statically-linked-library; neither works for me). It would be helpful to provide an example Dockerfile setup (e.g., using an official python image) that includes the appropriate install of sqlite3.

[nick-youngblut]

The last option in the Chroma docs on this issue:

If you are using a Debian based Docker container, older Debian versions do not have an up to date SQLite, please use bookworm or higher.

...seemed rather cryptic, at least to me, but after looking at the Chroma Dockerfile, I see that python:3.11-slim-bookworm is used instead of python:3.11-slim-buster.

However, when I switch from python:3.11-slim-buster to python:3.11-slim-bookworm, my docker build fails with the following error:

30.53 ERROR: Could not find a version that satisfies the requirement regex<2024.0.0,>=2023.12.25 (from crewai) (from versions: none)
30.53 ERROR: No matching distribution found for regex<2024.0.0,>=2023.12.25

[robink]

Same issue here, tried to build manually pysqlite3 but it didn't work either

[alexfazio]

Hello. I currently don't have a precise solution to your problem.

However, if you're aiming to utilize a PostgreSQL database, I recommend checking out this template notebook.

I hope it can provide you with some useful pointers.

[joaomdmoura]

The installation docs do not state that sqlite3 must be installed, in addition to running pip install crewai.

Great catch, we might have not added it as a dependency let me go ahead and do it.

[gromag]

Its's 16th of July and this still seems a problem. I just started with crew following the getting started and this still gives RuntimeError: Your system has an unsupported version of sqlite3. Chroma requires sqlite3 >= 3.35.0. 😞

Is there a solution, the Issue is still open.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.

[dregules]

This continues to be an issue for me, what's the solution? currently have in my requirements.txt

crewai crewai[tools] chromadb==0.4.0 pysqlite3-binary>=0.5.0

[theCyberTech]

Investigating

[theCyberTech]

This continues to be an issue for me, what's the solution? currently have in my requirements.txt

crewai crewai[tools] chromadb==0.4.0 pysqlite3-binary>=0.5.0

Hi @dregules,

I am struggling to reproduce this issue, is it possible you could send me your Dockerfile and reqs file?

[dregules]

@theCyberTech thanks for looking into it. I spent a good amount of time and the end result is that it was failing when i deployed only to streamlit cloud, which seems to have a known lower version of sqlite3 than what's required for chromadb dependency. I deplpyed to heroku and railway, and the issue resolved. For your reference, here's one of the main threads discussing this: https://discuss.streamlit.io/t/issues-with-chroma-and-sqlite/47950/5