Wheel install cause poetry.lock useless and misbehavior in docker deployment

[haicheviet]

Problems

Currently, docker deployment is using wheel for installing libs and dependency. If the dependency is small and easy, the current workflow is still fine but using wheel rather than poetry.lock cause poetry deployment philosophy is useless. For ex:

• File poetry.lock keep the consistency hash URL of dependency that will prevent you from automatically getting the latest versions of your dependencies. Easier to cache and security concern
• Not using poetry as full potential and may come with bugs in installation for multiple VM specs. For ex: PyTorch has multiple installs method for different specs (CPU and GPU), I can easily specify package version require in poetry but can not do the same with wheel package.

Solution propose

I think the wheel package install is not the way to go for python-monorepo deployment and we can use docker buildkit that supports docker context outside of the package install. Ref

Here is my current way to build docker for Monorepo: docker buildx build --build-context libs=../../libs .

Dockerfile

COPY --from=libs . ./../../libs
RUN poetry install # With share libs install: libcommon = {path = "../../libs/libcommon", develop = true}

[haicheviet]

I have re-implemented the Python monorepo to demonstrate my solution in a more well-packaged and ready-to-use way for this project. The improvements that I have made are as follows:

• Poetry playwell with docker container.
• Automated formatting and linting for the Python project.
• Docker-based CI/CD pipeline that allows sharing stages between libraries.
• Test and coverage page for whole monorepo including all subprojects using github action.

The project is now is on my GitHub at https://github.com/haicheviet/python-monorepo. If you find it useful, please star the project. Every star really makes my day!