FR: Can docker deployment be provided?

[1-2-3]

[mpadge]

FROM ubuntu:latest

RUN apt-get update -qq && apt-get install -y \
    git \
    python3 \
    python-is-python3 \
    pip \
    wget

RUN cd home \
    && git clone https://github.com/imartinez/privateGPT.git \
    && cd privateGPT \
    && pip install -r requirements.txt

# put these back as single line; split here for readability:
RUN echo \
"PERSIST_DIRECTORY=db\n
LLAMA_EMBEDDINGS_MODEL=models/ggml-model-q4_0.bin\n
MODEL_TYPE=GPT4All\n
MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin\n
MODEL_N_CTX=1000" > home/privateGPT/.env \
    && chmod a+x home/privateGPT/.env

RUN mkdir home/privateGPT/models \
    && cd home/privateGPT/models \
    && wget https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin \
    && wget https://huggingface.co/Pi3141/alpaca-native-7B-ggml/resolve/397e872bf4c83f4c642317a5bf65ce84a105786e/ggml-model-q4_0.bin

plus open ports and put an endpoint on if you want to query it, or just run directly and mount local volume into source_documents. (And thanks loads @imartinez for this awesome contribution!)

[imartinez]

That looks great, and really helpful. Do you think we should add a section for that on the Readme @mpadge? Maybe you could open a PR, would love to have you as contributor!

[Rots]

I see there's already a pull request https://github.com/imartinez/privateGPT/pull/120

[Deepansharora27]

Everytime I am trying to build an image using this Dockerfile, I get the Error Message that:

Not Enough Space Left on your Device

Even though I have ample of space present on my Host Machine. What should be the workaround for this ? Thanks !!

[Chuckame]

You know if it's possible to execute it in parallel ? (multi container instance using swarm, k3s/k8s, ...)

[drifter75]

I am getting below error:

#7 465.6 Successfully built llama-cpp-python pandoc olefile red-black-tree-mod compressed-rtf hnswlib python-docx python-pptx
#7 465.6 ERROR: Exception:
#7 465.6 Traceback (most recent call last):
#7 465.6   File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 165, in exc_logging_wrapper
#7 465.6     status = run_func(*args)
#7 465.6   File "/usr/lib/python3/dist-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
#7 465.6     return func(self, options, args)
#7 465.6   File "/usr/lib/python3/dist-packages/pip/_internal/commands/install.py", line 389, in run
#7 465.6     to_install = resolver.get_installation_order(requirement_set)
#7 465.6   File "/usr/lib/python3/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 188, in get_installation_order
#7 465.6     weights = get_topological_weights(
#7 465.6   File "/usr/lib/python3/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 276, in get_topological_weights
#7 465.6     assert len(weights) == expected_node_count
#7 465.6 AssertionError

[sammcj]

TBH I'd do something a bit more like:

### Build Image ###
FROM nvidia/cuda:11.8.0-devel-ubuntu22.04 as builder
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && \
  apt-get install -y --no-install-recommends python3 python3-dev python3-pip python3-venv git build-essential cmake

RUN git clone https://github.com/imartinez/privateGPT.git /app --depth 1

WORKDIR /app

ENV CMAKE_ARGS="-DLLAMA_OPENBLAS=on"
ENV FORCE_CMAKE=1

RUN --mount=type=cache,target=/root/.cache/pip LLAMA_CPP_PYTHON_VERSION="$(grep llama-cpp-python requirements.txt)" && \
  pip install --upgrade pip && \
  python3 -m venv venv && \
  . venv/bin/activate && \
  pip install "$LLAMA_CPP_PYTHON_VERSION" && \
  pip install --no-cache-dir -r requirements.txt

### Runtime Image ###
FROM nvidia/cuda:11.8.0-runtime-ubuntu22.04 as runtime

# Install Python and pip
RUN --mount=type=cache,target=/var/cache/apt apt-get update && \
  apt-get install -y --no-install-recommends \
  python3 python3-pip python3-venv && \
  rm -rf /var/lib/apt/lists/*

COPY --from=builder /app /app

RUN groupadd -g 1001 -o app && useradd -m -u 1001 -g 1001 -o -s /bin/bash app -d /app && \
  mkdir -p /models /db && \
  chown -R app:app /app /models /db

WORKDIR /app
USER app

RUN . venv/bin/activate

EXPOSE 5000
ENTRYPOINT ["python3", "/app/privateGPT.py"]

Might be missing a few things but that's close.