Fix docker compose up and MacBook segfault

[mbianchidev]

Implemented what was written on this comment and added some tweaks to make it work on without manual actions on the user's side

Could actually be a good idea to add a Dockerfile.mac instead of messing the Dockerfile.local file up, then it needs to be used in docker-compose.yaml when running on mac.

[imartinez]

The latest changes to dockerfiles merged address some of these issues, for example the download of models. Could you take a look and share your thoughts? https://github.com/imartinez/privateGPT/pull/1445

[mbianchidev]

The latest changes to dockerfiles merged address some of these issues, for example the download of models. Could you take a look and share your thoughts? #1445

@imartinez thanks a lot for checking in!

Docker compose still doesn't work out-of-the-box cause the model is not downloaded by default, the only way to do it without embedding it in the image (ofc not feasible) is an entrypoint script, also there are two changes not present also propedeutic to make it work for the dockerfile around poetry.

RUN poetry config installer.max-workers 10 RUN poetry install --extras chroma

And for settings to switch the vectorstore to chroma instead of qdrant

vectorstore:
  database: chroma

Also USER worker before running the python command seems also to be unnecessary

The goal of my contribution is to make the project smooth for docker and docker compose users, if you clone the repo and build image + run or just run docker compose up one should be able to have it running locally with no further action.

Let me know if I can implement any changes to match more with your vision!

[imartinez]

The latest changes to dockerfiles merged address some of these issues, for example the download of models. Could you take a look and share your thoughts? #1445

@imartinez thanks a lot for checking in!

Docker compose still doesn't work out-of-the-box cause the model is not downloaded by default, the only way to do it without embedding it in the image (ofc not feasible) is an entrypoint script, also there are two changes not present also propedeutic to make it work for the dockerfile around poetry.

RUN poetry config installer.max-workers 10 RUN poetry install --extras chroma

And for settings to switch the vectorstore to chroma instead of qdrant

vectorstore:
  database: chroma

Also USER worker before running the python command seems also to be unnecessary

The goal of my contribution is to make the project smooth for docker and docker compose users, if you clone the repo and build image + run or just run docker compose up one should be able to have it running locally with no further action.

Let me know if I can implement any changes to match more with your vision!

Thanks for the detailed explanation.

• "model is not downloaded by default" -> there is already an entrypoint setup script that you can run after building the image that will take care of that. I think that's the best way to go: docker compose run --rm --entrypoint="bash -c '[ -f scripts/setup ] && scripts/setup'" private-gpt
• poetry max worker is something we should add, I agree
• --extras chroma should not be added by default. Qdrant is the default vector db, and therefore Chroma-related dependencies are optional. Same for the change you propose to the settings.yaml file.
• "USER worker" not sure about that one. I could test it out.

Let me know your thoughts.

[mbianchidev]

Thanks for the detailed explanation.

• "model is not downloaded by default" -> there is already an entrypoint setup script that you can run after building the image that will take care of that. I think that's the best way to go: docker compose run --rm --entrypoint="bash -c '[ -f scripts/setup ] && scripts/setup'" private-gpt
• poetry max worker is something we should add, I agree
• --extras chroma should not be added by default. Qdrant is the default vector db, and therefore Chroma-related dependencies are optional. Same for the change you propose to the settings.yaml file.
• "USER worker" not sure about that one. I could test it out.

Let me know your thoughts.

I think that the entrypoint as proposed is a bit verbose but this may be only my own preference.

I had to do a bit of research before getting up and running with compose and judging by some recent and similar issues opened I was not the only one encountering the errors. What about adding the command: docker compose run --rm --entrypoint="bash -c '[ -f scripts/setup ] && scripts/setup'" private-gpt to both the readme file of the repo and the doc?

A middle ground could also be creating a Dockerfile.mac to avoid touching the original one at all, what do you think?

Chroma is the only one working on mac/local if I'm not mistaken, correct? If that's the case I do think it might be better to enable it by default, what would the consequences be? Ofc it's a trade-off!

The deletion of USER worker was a test I made and it brought me to run docker compose successfully on top of not having any segfault error on mac (M2 chip), please do test it out and let me know the result.

Thanks again for engaging and keep up the good work!

[github-actions[bot]]

Stale pull request