AutoGPT
AutoGPT copied to clipboard
Significant-Gravitas
Docker build fails due to sorcery dependency
Duplicates
- [X] I have searched the existing issues
Steps to reproduce 🕹
$ docker build -t autogpt .
Current behavior 😯
[+] Building 5.2s (10/11)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 688B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/python:3.11-slim 1.2s
=> [1/7] FROM docker.io/library/python:3.11-slim@sha256:286f2f1d6f2f730a44108656afb04b131504b610a6cb2f341391 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 1.55kB 0.0s
=> CACHED [2/7] RUN useradd --create-home appuser 0.0s
=> CACHED [3/7] WORKDIR /home/appuser 0.0s
=> CACHED [4/7] RUN chown appuser:appuser /home/appuser 0.0s
=> CACHED [5/7] COPY --chown=appuser:appuser requirements.txt . 0.0s
=> ERROR [6/7] RUN pip install --no-cache-dir --user -r requirements.txt 3.9s
------
> [6/7] RUN pip install --no-cache-dir --user -r requirements.txt:
#10 0.978 Collecting beautifulsoup4
#10 1.095 Downloading beautifulsoup4-4.12.2-py3-none-any.whl (142 kB)
#10 1.109 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.0/143.0 kB 11.2 MB/s eta 0:00:00
#10 1.133 Collecting colorama==0.4.6
#10 1.142 Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
#10 1.163 Collecting openai==0.27.2
#10 1.173 Downloading openai-0.27.2-py3-none-any.whl (70 kB)
#10 1.177 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.1/70.1 kB 259.8 MB/s eta 0:00:00
#10 1.194 Collecting playsound==1.2.2
#10 1.204 Downloading playsound-1.2.2-py2.py3-none-any.whl (6.0 kB)
#10 1.224 Collecting python-dotenv==1.0.0
#10 1.237 Downloading python_dotenv-1.0.0-py3-none-any.whl (19 kB)
#10 1.275 Collecting pyyaml==6.0
#10 1.286 Downloading PyYAML-6.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (732 kB)
#10 1.314 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 732.4/732.4 kB 28.3 MB/s eta 0:00:00
#10 1.328 Collecting readability-lxml==0.8.1
#10 1.336 Downloading readability_lxml-0.8.1-py3-none-any.whl (20 kB)
#10 1.365 Collecting requests
#10 1.373 Downloading requests-2.28.2-py3-none-any.whl (62 kB)
#10 1.377 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 36.8 MB/s eta 0:00:00
#10 1.398 Collecting tiktoken==0.3.3
#10 1.409 Downloading tiktoken-0.3.3-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.6 MB)
#10 1.457 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 35.8 MB/s eta 0:00:00
#10 1.473 Collecting gTTS==2.3.1
#10 1.481 Downloading gTTS-2.3.1-py3-none-any.whl (28 kB)
#10 1.506 Collecting docker
#10 1.515 Downloading docker-6.0.1-py3-none-any.whl (147 kB)
#10 1.522 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 147.5/147.5 kB 31.0 MB/s eta 0:00:00
#10 1.539 Collecting duckduckgo-search
#10 1.547 Downloading duckduckgo_search-2.8.5-py3-none-any.whl (30 kB)
#10 1.587 Collecting google-api-python-client
#10 1.597 Downloading google_api_python_client-2.85.0-py2.py3-none-any.whl (11.2 MB)
#10 1.914 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.2/11.2 MB 36.2 MB/s eta 0:00:00
#10 1.959 Collecting pinecone-client==2.2.1
#10 1.968 Downloading pinecone_client-2.2.1-py3-none-any.whl (177 kB)
#10 1.974 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 177.2/177.2 kB 42.4 MB/s eta 0:00:00
#10 2.002 Collecting redis
#10 2.013 Downloading redis-4.5.4-py3-none-any.whl (238 kB)
#10 2.020 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 238.9/238.9 kB 40.3 MB/s eta 0:00:00
#10 2.141 Collecting orjson
#10 2.149 Downloading orjson-3.8.10-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (261 kB)
#10 2.158 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 261.1/261.1 kB 36.1 MB/s eta 0:00:00
#10 2.284 Collecting Pillow
#10 2.295 Downloading Pillow-9.5.0-cp311-cp311-manylinux_2_28_aarch64.whl (3.2 MB)
#10 2.382 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 37.0 MB/s eta 0:00:00
#10 2.413 Collecting selenium
#10 2.423 Downloading selenium-4.8.3-py3-none-any.whl (6.5 MB)
#10 2.602 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 37.0 MB/s eta 0:00:00
#10 2.626 Collecting webdriver-manager
#10 2.634 Downloading webdriver_manager-3.8.6-py2.py3-none-any.whl (27 kB)
#10 2.816 Collecting coverage
#10 2.830 Downloading coverage-7.2.3-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (232 kB)
#10 2.832 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.4/232.4 kB 140.7 MB/s eta 0:00:00
#10 2.856 Collecting flake8
#10 2.864 Downloading flake8-6.0.0-py2.py3-none-any.whl (57 kB)
#10 2.868 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.8/57.8 kB 45.6 MB/s eta 0:00:00
#10 2.982 Collecting numpy
#10 2.997 Downloading numpy-1.24.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
#10 3.551 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 23.1 MB/s eta 0:00:00
#10 3.611 Collecting pre-commit
#10 3.621 Downloading pre_commit-3.2.2-py2.py3-none-any.whl (202 kB)
#10 3.636 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 202.7/202.7 kB 14.6 MB/s eta 0:00:00
#10 3.664 Collecting black
#10 3.673 Downloading black-23.3.0-py3-none-any.whl (180 kB)
#10 3.691 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.0/181.0 kB 11.2 MB/s eta 0:00:00
#10 3.707 ERROR: Ignored the following versions that require a different python version: 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11
#10 3.707 ERROR: Could not find a version that satisfies the requirement sourcery (from versions: none)
#10 3.707 ERROR: No matching distribution found for sourcery
#10 3.763
#10 3.763 [notice] A new release of pip available: 22.3.1 -> 23.0.1
#10 3.763 [notice] To update, run: pip install --upgrade pip
------
executor failed running [/bin/sh -c pip install --no-cache-dir --user -r requirements.txt]: exit code: 1
Expected behavior 🤔
Docker build does not fail
Your prompt 📝
N/A
@kkyr I was able to resolve this by updating the following line in Dockerfile:
# original
RUN pip install --no-cache-dir --user -r requirements.txt
# updated
RUN python3 -m pip install --no-cache-dir --user -r requirements.txt
The same can be applied to .devcontainer.json
// original
"postCreateCommand": "pip3 install --user -r requirements.txt",
// updated
"postCreateCommand": "python3 -m pip install --user -r requirements.txt",
This is a workaround and I'm trying to identify the root cause.
I hope that helps!
@kkyr I pulled the latest changes and tried docker build it myself. Looks fine. Can you test build it after git fetch or git pull the latest changes please?
@Qoyyuum I've also tried using different base images to see if that resolves the issue with no luck. Normally I'd say this is an issue with my local python install, but since this is a docker build it's not relevant (I think). Curious if you have any ideas?
Having same issues also after switching to python3 -m option, removed sourcery requirement for now
I pulled latest changes and tried the workarounds mentioned above but still seeing the same issue.
If I build without the sourcery requirement the build works but I'm running into #1513 when running the next command.
It's also breaking my docker compose for auto-gpt
#0 81.12 ERROR: Could not find a version that satisfies the requirement sourcery (from versions: none)
#0 81.12 ERROR: No matching distribution found for sourcery
------
Dockerfile:27
--------------------
26 |
27 | >>> RUN apk update && apk add --no-cache git vim curl pkgconfig \
28 | >>> cairo-dev g++ gcc libffi-dev libjpeg-turbo-dev libxml2-dev libxslt-dev \
29 | >>> gstreamer gst-plugins-base gst-plugins-good \
30 | >>> girara-dev gobject-introspection-dev sudo wget curl \
31 | >>> && addgroup -g 1000 -S auto-gpt && adduser -u 1000 -S -G auto-gpt auto-gpt \
32 | >>> && pip install --upgrade pip \
33 | >>> && pip install pycairo PyGObject \
34 | >>> && python3 -m pip install --no-cache-dir --user -r requirements.txt \
35 | >>> && chown -R 1000:1000 /app
36 |
--------------------
ERROR: failed to solve: process "/bin/sh -c apk update && apk add --no-cache git vim curl pkgconfig cairo-dev g++ gcc libffi-dev libjpeg-turbo-dev libxml2-dev libxslt-dev gstreamer gst-plugins-base gst-plugins-good girara-dev gobject-introspection-dev sudo wget curl && addgroup -g 1000 -S auto-gpt && adduser -u 1000 -S -G auto-gpt auto-gpt && pip install --upgrade pip && pip install pycairo PyGObject && python3 -m pip install --no-cache-dir --user -r requirements.txt && chown -R 1000:1000 /app" did not complete successfully: exit code: 1
The root cause here is that recently sourcery has been added as a dependency, which only supports 64bit python:
... Currently, we only support 64-bit.
Are you using a 32-bit machine? If not, could you please try installing Python 64-bit and checking if
pip install sourcery-cliworks?
Originally posted by @ruancomelli in https://github.com/sourcery-ai/sourcery/issues/257#issuecomment-1194539132
Changing the base image in Dockerfile fixed the issue for me:
FROM amd64/python:3.11-slim
looks like aarch64 isn't supported either, is there a way to make sourcery an optional dependency?
@mihalycsaba You can simply remove it from the requirements file, then re-run a pip install. It's not a breaking-dep.
Removing sourcery from the requirements list directly leads to #1624 , for me, while I am trying to execute a Dockerfile. Although the error is different in a linux environment (too lengthy for my term to capture).
@mihalycsaba You can simply remove it from the requirements file, then re-run a pip install. It's not a breaking-dep.
Doesn't work well with auto-build tools
Removing sourcery from the requirements list directly leads to #1624 , for me, while I am trying to execute a Dockerfile.
Same here, for my project at https://github.com/claytondukes/autogpt-docker The other problem is that sourcery only works on amd64, so no Mac M1 support?
After some hacking away I came away building a container and, possibly beyond the scope of this repo, adding gotty. And before I forget, let me give credit where it is due if this is actually successful. All of my efforts were influenced by @claytondukes's implementation and the specific dockerfile config from here. And of course asking GPT to do the heavy lifting.
Note: An assumption I am making here is that you have already cloned the repo into the current working directory, and you are simply overwriting the files currently available in the repo.
First requirements-docker.txt needed an additional dependency to avoid an error claiming a missing "git" package:
#original
<rest of file>
black
isort
#new
<...>
black
isort
gitpython #add this one
Second, explain to docker how to build the image. You can completely copy and paste this over the current repo's Dockerfile Dockerfile:
### Let's get Got(ty)
ARG GOTTY_VERSION=v1.5.0
# Builder stage
FROM debian:bullseye-slim AS builder
ARG GOTTY_VERSION
WORKDIR /build
ADD https://github.com/sorenisanerd/gotty/releases/download/${GOTTY_VERSION}/gotty_${GOTTY_VERSION}_linux_arm64.tar.gz gotty-aarch64.tar.gz
ADD https://github.com/sorenisanerd/gotty/releases/download/${GOTTY_VERSION}/gotty_${GOTTY_VERSION}_linux_amd64.tar.gz gotty-x86_64.tar.gz
# Determine which executable we need, and extract the bin file
RUN tar -xzvf "gotty-$(uname -m).tar.gz"
# Let's get AutoGPT now
FROM python:3.11-slim
# Copy Gotty from builder stage
COPY --chmod=+x --from=builder /build/gotty /bin/gotty
# Install git
RUN apt-get -y update
RUN apt-get -y install git
# Set environment variables
ENV PIP_NO_CACHE_DIR=yes \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
# Create a non-root user and set permissions
RUN useradd --create-home appuser
WORKDIR /home/appuser
RUN chown appuser:appuser /home/appuser
USER appuser
# Copy the **MODIFIED** requirements.txt file and install the requirements
COPY --chown=appuser:appuser requirements-docker.txt .
RUN pip install --no-cache-dir --user -r requirements-docker.txt
# Copy the application files
COPY --chown=appuser:appuser autogpt/ autogpt
# Expose Gotty exposing AutoGPT's terminal responses.
EXPOSE 8080
# Inject the required OPENAI API KEY as an environment variable.
ENV OPENAI_API_KEY=${OPENAI_API_KEY}
# Set the entrypoint
ENTRYPOINT ["/bin/gotty", "--port", "8080", "--permit-write", "--title-format", "AutoGPT Terminal", "python", "-m", "autogpt"]
Build & Run
docker build . -t autogpt
docker run -p 8080:8080 -e OPENAI_API_KEY=<openai-api-key> --name=autogpt autogpt
Not perfect and could be heavily revised but hoping I can hand this off to someone much smarter hah.
Not perfect and could be heavily revised but hoping I can hand this off to someone much smarter hah.
Man, I went to update my repo today based on the recent changes to Auto-GPT. It's not even been a week and all my work is outdated! lol This project is moving way too fast for me to keep up.
Edit: If anyone wants to take over my project at https://github.com/claytondukes/autogpt-docker please reach out. I wish I had more time, but have to work my day job and maintain family time.
I can't help but wonder if it's possible to run gotty or ttyd in it's own docker service using docker-compose and then share the command prompt from the auto-gpt service to the other "tty" service. This seems like it would nicely decouple the two while also adding a way to access auto-gpt from a web browser?
