chore(docker): dockerfiles improvements and fixes

[qdm12]

• UID and GID build arguments for worker user. This is useful to not have to change permissions on bind mounted directories.
• POETRY_EXTRAS build argument with default values. Useful to re-build the image with some extras using --build-arg POETRY_EXTRAS="..."
• Copy Makefile in Docker image so commands such as make ingest can be used
• Do NOT copy markdown files. I doubt anyone reads a markdown file within a Docker image
• Fix PYTHONPATH value where path should be /home/worker/app/private_gpt instead of /private_gpt
• Set home directory to /home/worker when creating user. This resolves a bunch of side dependencies warnings/problems.
• Combine ENV instructions together. Just a tiny bit faster to rebuild.
• Define environment variables with their defaults
  • For documentation purposes
  • Reflect defaults set in settings-docker.yml
• PGPT_EMBEDDING_MODE to define embedding mode. It's not always the same mode as the llm.
• Remove ineffective python3 -m pipx ensurepath (does not work and is fulfilled by ENV PATH="/root/.local/bin:$PATH")
• Use && instead of ; to chain commands to detect failure better

Next PRs (after this one gets merged):

1. merge both Dockerfile together. Highly possible especially given the new build argument POETRY_EXTRAS.
2. Add missing openai variables and definitions in settings-docker.yaml
3. Add docker documentation
4. Pick up #1428

[szastavnyy]

Hello All. Please clarify what command is better to use to build docker image locally, upload to GCP and use ? I do not see useful info in documentation related to docker creation

[qdm12]

@szastavnyy see https://github.com/zylon-ai/private-gpt/issues/1789#issuecomment-2031351986

[qdm12]

@zylon-ai I keep on updating this branch, it would be nice to have a review at some point. Thanks 😉

[github-actions[bot]]

Published docs preview URL: https://privategpt-preview-4d7135ba-2e40-4816-8fa3-654844aaa338.docs.buildwithfern.com

[jaluma]

Updated this PR since it's really useful :) Goal: Implement a better out-of-the-box Docker experience.

---

I've included:

1. Fix Tiktoken cache https://github.com/zylon-ai/private-gpt/issues/1876#issuecomment-2138500112
2. Change Docker-compose port to 8001
3. Autopulling images when docker image is running
4. All requested change by myself

[github-actions[bot]]

Published docs preview URL: https://privategpt-preview-0b4b3e54-d4ea-42e4-ab4a-2794943cf525.docs.buildwithfern.com

[qdm12]

Thanks, sorry for the delay getting back at this PR! I'll rebase other branches I have based off this one soon, for more Docker-related improvements/ease-of-use.