[FEATURE] Use the source code from the repo for Docker images

[aries1980]

Problem

I'd like to test changes in isolation. Docker is an ideal candidate to do so.

Solution

In order to do have the current source code, we need it to COPY into the Docker image (maybe pip install . ?) instead of using the latest tagged version.

[samschott]

Isn't this handled already by the nightly builds from the dev branch? Or do you mean building on every commit?

It would also be nice to get docker images for all PRs. The upload to docker hub currently fails for external PRs because the github secrets the the dockerhub password are not accessible. A way around this would be to run the action on "pull_request_target".

[aries1980]

I'm afraid I forgot to think about it. Yes, we have nightly builds, but the Github Action I made checks for the latest tagged version. 🤦‍♂️

Yes, I can include the PRs to be made and pushed, so we can try it.

The upload to docker hub currently fails for external PRs because the github secrets the the dockerhub password are not accessible.

I mentioned this in my PR about the Docker image. You need to add the 3 secrets in the settings of this Git repository. Only the maintainers of this repo can do it, I can't.

The 3 secrets are:

• DOCKERHUB_USERNAME
• DOCKERHUB_PASSWORD
• DOCKERHUB_TOKEN

[samschott]

You need to add the 3 secrets in the settings of this Git repository.

I've added them. PRs from maintainers are pushed to Docker Hub correctly. The issue is that such secrets are not available to PRs from external users because they could potentially modify the github action to print the secrets or send them to some server.

I think the proper solution would be to run the action on pull_request_target so that the action of the target branch will be executed on a PR. This action does get access to secrets defined for the repo while ensuring that the action itself is not modified. It can then check out the PR's code and upload it to docker hub. Secrets never get exposed to any potentially malicious code.

Edit: This would indeed require checking out the code with the desired SHA and using a pip install ..

[pataquets]

@samschott I'd like to chime in to provide a few insights on the issue, based on my own experience. The current Dockerfile just installs via pip some files built outside of Docker into the image. It's not a source build (which is the most common use and the common practice with Docker).

An official, built-in Dockerfile which allows to build a Docker image directly from source would be more useful. Case in point: I wanted to test latest HEAD and start hacking around and, since building is quite an involved process for me and my Python host OS' packages are outdated, I looked for a Dockerfile to build an image and attempted to build directly from the master branch:

docker build --force-rm -t maestral github.com/samschott/maestral#master

Unfortunately, since it's not a source build and it expects the binary to be "built" outside of the Docker workflow, it didn't work for me.

Notice you can easily test PRs with a one-liner:

docker build --force-rm -t maestral github.com/samschott/maestral#pull/ID/head

Potential use-cases for a source build Dockerfile:

• Running: With a proper Dockerfile, you could just issue a docker build -t maestral github.com/samschott/maestral#master to locally build a Docker image ready to run with a simple docker run. No build tools needed, no packaging, not even cloning the repo! And it would be straight from source (any desired branch or tag).
• Development: You can start hacking right away with just Docker installed without needing any further toolchain on your system, since the only thing needed to test your code is docker build ....
• Distribution: You can use the same workflow which works locally to build the images and push them to Docker Hub (even for other architectures) with a simple Github Action. Thus, you'll have a public Docker image with the latest code on HEAD (aka. daily builds).
• Documentation: since the Dockerfile should install all required build tools, libraries and other packages and would require also to run all the required build commands, it would serve as reference to the build process (or even as the canonical source of truth for installation if you approach this in a "the code are the docs" fashion). Just my 2c.