Docker build feedback

[idnovic]

Greetings,

please add a docker hub build. So that we can update based on release tags.

Thank you.

I want to try out mox later. It looks exactly like what I have been looking for.

[mjl-]

I don't mind making a docker image. There is a reason I haven't done so yet: I don't know if typical docker-based deploys will get access to the actual IPs of the machine and of incoming connections.

For spam filtering, it is important to know which server connected. It's good to know the IP of the machine, to check if it occurs in DNS block lists, and to ensure only the configured IPs are used for outgoing connections (that have proper reverse DNS and SPF configured).

I can do a test with a basic docker-compose. How will this work for deploying in kubernetes? I used docker swarm in the past, and think all connections went through a local NAT, hiding the IPs from remote.

[idnovic]

Ahh I see. I think you can expect a reverse proxy with ssl termination in front of the web admin ui.

My usecase is with podman instead of docker. The container runs inside a private net with internet access over a gateway.

The needed ports for mail traffic are bound from the docker/podman container to an interface.

In the case of a mailserver one would choose 0.0.0.0 or the ip of the server.

Basically if you do the port mapping correctly, then mox is directly connected to the internet for the mail ports but not for the admin interface.

The reasoning behind that is the reverse proxy which serves ssl and forwards traffic to all other containers as well. Also port based.

Let me add an example below...

[idnovic]

Let's specify the following for the example:

1.2.3.4 is the server public ip 0.0.0.0 bind to all internal/external interfaces 127.0.0.1 binds to localhost 10.90.0.0/16 is a private net within the server. In this case provided by an network interface of podman.

A normal container without port mappings does not expose ports and is inside the private 10.90.0.0/16 net. DNS resolution can be enabled. In this case the container could access web resources but not the other way around over the 10.90.0.1 gateway.

But it is possible to bind ports from container to interface. I can bind ports from the container 10.90.0.5 to the gateway 10.90.0.1. That is useful if you have multiple private nets for groups of containers each privately talking but not exposed to each other.

Or I bind ports globally. On the servers ip address or 0.0.0.0 to bind to all routable addresses.

In the first example NAT would be used. It would be possible to forward the right originating ip per reverse proxy to the internal server. But that is only useful for web servers.

In the second example the traffic gets forwarded but it is transparent.

[idnovic]

borrowed from maddy:

EXPOSE [25 143 993 587 465]
ENTRYPOINT ["/bin/maddy", "-config", "/data/maddy.conf"]
CMD ["run"]

This would work in most setups. An exposed port is public. If someone wants to expose the same port only locally, they need to bind to a specific ip/interface anyway.

That said I am unsure about the docker desktop tools and such. I also do not have knowledge about kubernetes. I would expect that the people running complex clusters know how to map the ports correctly.

[tim-hub]

Let's specify the following for the example:

1.2.3.4 is the server public ip 0.0.0.0 bind to all internal/external interfaces 127.0.0.1 binds to localhost 10.90.0.0/16 is a private net within the server. In this case provided by an network interface of podman.

A normal container without port mappings does not expose ports and is inside the private 10.90.0.0/16 net. DNS resolution can be enabled. In this case the container could access web resources but not the other way around over the 10.90.0.1 gateway.

But it is possible to bind ports from container to interface. I can bind ports from the container 10.90.0.5 to the gateway 10.90.0.1. That is useful if you have multiple private nets for groups of containers each privately talking but not exposed to each other.

Or I bind ports globally. On the servers ip address or 0.0.0.0 to bind to all routable addresses.

In the first example NAT would be used. It would be possible to forward the right originating ip per reverse proxy to the internal server. But that is only useful for web servers.

In the second example the traffic gets forwarded but it is transparent. @idnovic Thanks for the sharing,

Docker build will be great!

@mjl- am sure, anything work in docker compose, will work in k8s. I would like to help on that to try on my own k8s cluster, if there is a compose file.

[wp-coin]

you only need to specify the port number

services:
  frontend:
    image: awesome/webapp
    ports:
      - "443:8043"
    networks:
      - front-tier
      - back-tier
    configs:
      - httpd-config
    secrets:
      - server-certificate

https://docs.docker.com/compose/compose-file/

[mjl-]

Excellent, thanks for the examples. Going to work on this.

To start mox you need a config. Currently the recommended way is to run the quickstart. Is that a common enough thing to do with docker deploys? It would probably require running "docker run mox quickstart ..." once before starting mox.

I started pondering about changing mox to create a mostly empty config when started for the first time (when no config file exists yet), without any domain configured, with just IPs and hostname detected and configured, then start serving the admin interface where you can add the first domain/account and where you'll see the required DNS records.

[wp-coin]

creating a compose file with the services that are required, and then specify in compose file to run mox quickstart

I would say, this is the preferred way to launch a docker image

[mjl-]

Thanks, I added the quickstart instructions to the new docker-compose file.

Images will be at moxmail/mox. See https://hub.docker.com/r/moxmail/mox. It currently only has a tag "test" with a docker image based on the latest commit that added the docker-compose file. I used a slightly earlier version to set up a mox on a clean vps.

A few more notes:

• Only docker images for linux/amd64 for now. My machine (debian testing) does not have the "docker buildx" command for building for other platforms. Let's wait for folks to request linux/arm(64) and other platforms before providing them. Are there better ways to create images for other platforms?
• In the near-future I want to change mox to start as root, bind to privileged network ports, then drop to an unprivileged mox user. So the setcap in the Dockerfile will go away then, and the quickstart instructions will change. The file permissions will probably stay the same.
• The "go build" in the docker file is not built in module-mode in the sense that the tag/version number is included in the runtime module information. If anyone knows how that is commonly done, let me know. For now I'll probably build with an explicit "go install github.com/mjl-/mox@v..." when making docker images. Ideally a system similar to https://beta.gobuilds.org that generates docker images on demand would be used (reproducible), but let's leave that for another day.

Could someone have a look and see if the Dockerfile and docker-compose.yml adhere to the docker way of doing things?

And could someone try moxmail/mox:test to set up mox with docker-compose?

[adamshand]

This is super helpful, thank you! Just learned about Mox and trying it out now. :-)

[tim-hub]

Thanks, I added the quickstart instructions to the new docker-compose file.

Images will be at moxmail/mox. See https://hub.docker.com/r/moxmail/mox. It currently only has a tag "test" with a docker image based on the latest commit that added the docker-compose file. I used a slightly earlier version to set up a mox on a clean vps.

A few more notes:

• Only docker images for linux/amd64 for now. My machine (debian testing) does not have the "docker buildx" command for building for other platforms. Let's wait for folks to request linux/arm(64) and other platforms before providing them. Are there better ways to create images for other platforms?
• In the near-future I want to change mox to start as root, bind to privileged network ports, then drop to an unprivileged mox user. So the setcap in the Dockerfile will go away then, and the quickstart instructions will change. The file permissions will probably stay the same.
• The "go build" in the docker file is not built in module-mode in the sense that the tag/version number is included in the runtime module information. If anyone knows how that is commonly done, let me know. For now I'll probably build with an explicit "go install github.com/mjl-/mox@v..." when making docker images. Ideally a system similar to https://beta.gobuilds.org that generates docker images on demand would be used (reproducible), but let's leave that for another day.

Could someone have a look and see if the Dockerfile and docker-compose.yml adhere to the docker way of doing things?

And could someone try moxmail/mox:test to set up mox with docker-compose?

I do have a server run on arm64, I could try to help to build that if need.

[idnovic]

I changed the issue to be about docker feedback.

I would recommend to add environment variables to mox. Currently mox works like most single bin applications. The configuration is created during the first usage and provided by command flags.

In the container world you do not drop to the shell for the configuration.

Two suggestions:

-Let us mount a config file If file exists load the config. If file does not exist create it. But the config file is always mounted from the host to inside the container. Do not keep the configuration container bound.

A new mox release should always trigger a complete reinstall of mox. So we need an external config. And mox needs to be aware of it. No need to retrigger first time setup routines during an container update.

-Let us specify env variables Mox container should read the environment variables and create a config file based on them if no config file exists. If it exists the environment variables come before the settings in the file.

That means first look for config file second check env variables third merge config and write it out to the config file. Env takes priority before conf file as long as they defer.

[idnovic]

--volume /xyz/mox:/config:rw \
--env domain=mydomain.com \

Example for the configuration. mox is already mounting a conf directory. Missed it before.

[mjl-]

So I think the first suggestion, about mounting the config file is covered? The quickstarts creates the two config files in your local config/ directory. I like to keep my configs in git, which works nicely, even though mox can rewrite the domains.conf file.

For the second suggestion, specifying env variables. I feel it is a bit tricky, because you do need to read the output of the quickstart and follow the instructions. We could run the quickstart automatically, but I feel it doesn't really help the user. Perhaps it can work in a future where the quickstart sets up less (no domain yet), and you continue configuration through the web interface. Though I think users should still be pushed to look at the generated config with the IPs and the services that are enabled on them.

[mjl-]

@tim-hub

I do have a server run on arm64, I could try to help to build that if need.

Have you been creating docker images on another system (amd64) to run on arm64? And are you doing it with "docker buildx" or another tool? Perhaps I should just go and install that buildx tool from docker directly...

[idnovic]

Perhaps it can work in a future where the quickstart sets up less (no domain yet), and you continue configuration through the web interface

A web interface sounds good. If environment variables are not feasible that would be fine too. I was comparing to the other container applications we use. To be able to provide every needed config input during creation is a nice to have feature.

[mjl-]

I want to implement the simpler-quickstart-and-create-domain-afterwards soon, I think it will make setting mox up easier.

But first the docker images. Latest update is that I think I managed to build multi-platform images (using cross-compilation in a docker image and using docker build --platform ...; had to switch to podman), with the correct go module versions stamped into the binary. Starting as root and dropping privileges is almost ready too. Need to figure out what the right file permissions should be, and document the upgrade path (too bad mox already needs one now, but I'm figuring the sooner the better). Also need to do some testing...

[idnovic]

Since you mentioned podman, I am unsure what a good test platform could be for mox. Podman has 4 network backends: -cni (old) for root -netavark (current) for root -slirp4netns (current) for root-less -passt/pasta (new) for root-less

I want to try it with pasta because of this by default, addresses and routes are copied from the host, that is, container users will see the same IP address and routes as if they were in the init namespace context. The interface name is also sourced from the host upstream interface with the first default route in the routing table. This is also configurable as documented

[mjl-]

with the additional docker build fix at https://github.com/mjl-/mox/commit/f3f2c6f8ea90594d0b2908a3622ecde5b4f7bd5a i think the docker images should now work, including cross-platform.

see https://hub.docker.com/r/moxmail/mox/tags

this also includes the recent commit that makes mox startup as root, then drop to unprivileged user. the docker-compose.yml file has updated instructions. i think the update should be a matter of getting the updated docker-compose.yml, updated docker image and restart.

i'm not experienced with these docker manifests. i'm assuming that pushing to an existing manifest overwrites the images inside, instead of appending. so that pushing a new :latest will set new images.

[tim-hub]

@tim-hub

I do have a server run on arm64, I could try to help to build that if need.

Have you been creating docker images on another system (amd64) to run on arm64? And are you doing it with "docker buildx" or another tool? Perhaps I should just go and install that buildx tool from docker directly...

I haven't but my server and my laptop is arm64, buildx sounds better for this, it seems like don't require arm64 cpu to build arm64 image then

[sielnet]

Here is the podman test from me @idnovic Podman guide:

mox needs a mox.conf file ready before you start the container. Simply download the mox go binary locally and run ./mox quickstart [email protected]. Take note of the terminal output. You will need the admin password and DNS settings. Now copy the generated directories data; config to your container mount location ${HOME}/containers/mox/.

Create the mox pod with these commands:

## pod mox
  podman pod create \
    --replace \
    --name mox \
    --network-alias=mox \
    --memory=512m \
    --publish PUBLIC_IP:25:25 \
    --publish PUBLIC_IP:465:465 \
    --publish PUBLIC_IP:587:587 \
    --publish PUBLIC_IP:993:993 \
    --publish PUBLIC_IP:143:143 \
    --publish PUBLIC_IP:80:80 \
    --publish PUBLIC_IP:443:443
    
# mox-mail
  podman run \
    --pod mox \
    --detach \
    --replace \
    --restart=unless-stopped \
    --name=mox-mail \
    --label io.containers.autoupdate=registry \
    --volume ${HOME}/containers/mox/config:/mox/config:Z \
    --volume ${HOME}/containers/mox/data:/mox/data:Z \
    docker.io/moxmail/mox:v0.0.2-0.20230227111955-92e018e46371

Note: If you run podman as root you need to set a user in mox.conf. The default mox user is not needed. Instead you can provide the uid of a local user as interger like 1000. Now you just need to chown the mox mount location.

Functionality depends on the settings you have made in the mox.conf file.

[sielnet]

Bugs:

The lastest tag of the docker build does not work.

After you create a docker/podman container of mox it keeps crashing because the mox.conf file is missing. Please allow to run the quickstart via a ENV variable. --env [email protected]. And output the terminal output of mox to a quickstart.done file. Users could read the important information from this file.

I was not able to run mox behind a reverse proxy. I received a http 400 error for the admin web ui and I was not able to setup ssl for mta-sts at all.

I think the problem is that I am not able to forward a tcp connection with the reverse proxy because I need to forward it based on subdomain. Like mail. but tcp has no concept of subdomain. I could fix it if I map the mail subdomain on an other IP than the root domain.

This is a very specific issue. Nothing you can do. But I think it is not possible to run mox behind a reverse proxy. It should work if your server has multiple IPs and a dedicated IP for mail. Or you need to disable mta-sts and other features that require port 443. Basically everything but the user/admin interface.

Problem described with Nginx reverse proxy: -domain has 1 IP -server uses this IP -root domain and www already use port 443 -a reverse proxy is able to forward http requests -mox user ui and admin ui can be forwarded with the reverse proxy mail.subdomain.com/admin -port 80 of the subdomain mail. can be forwarded to the mox container with a proxy_pass config -port 443 can not be forwarded because it is already in use for root domain and www It it not possible to use proxy_pass for port 443. Instead we need to use Stream pass. Stream pass proxies the request to the SSL port of mox and even forwards the SSL certificate from mox to the user. But the problem is that Stream pass can not be used. The port 443 is already in use for root domain and www That is the reason we need a second IP address. To be able to bind port 443 for the Stream pass config.

[mjl-]

i created a new docker image with the latest changes: docker.io/moxmail/mox:v0.0.2-0.20230303234902-15e262b0431d-go1.20.1-alpine3.17.2

@sielnet i'll create the latest tag when there is something that is known to work for folks. probably will be together with a release of v0.0.2, hopefully the day after tomorrow.

@sielnet if mox stops due to a missing config file, then the mount of config/ may not have worked? why can't you run quickstart manually with an equivalent of "docker-compose run" or "docker run"?

i have been able to setup & run mox with the docker-compose.yml file from the repo. doesn't podman have a compose-variant? i haven't tried running this with podman yet btw.

there is now also an option to the quickstart, -existing-webserver, to run with an existing webserver. it writes a different config file, and i made some more fixes. it was certainly quite tricky to configure correctly, but should be much easier with this flag.

i also added webserver functionality to mox, so it can function as reverse proxy and serve files. so hopefully that takes away most of the reason to want to keep another server running on port 443/80, it complicates matters quite a bit. if running from docker, you still need to give mox access to your host network, running with NATed traffic won't work.

@tim-hub the podman script used to build docker images is docker-release.sh in the repo. i'm running it on my linux/amd64 machine.

[mjl-]

i replaced use of docker.io (hub.docker.com) with a self-hosted registry at https://r.xmox.nl/. let me know if this is causing issues, it is a new registry implementation...

[tim-hub]

i replaced use of docker.io (hub.docker.com) with a self-hosted registry at https://r.xmox.nl/. let me know if this is causing issues, it is a new registry implementation...

just qurious which registry UI is it? of https://r.xmox.nl/

[mjl-]

just qurious which registry UI is it? of https://r.xmox.nl/

https://github.com/mjl-/vex, the html is in {index,repo,manifest}.html and generated by html.go

[mjl-]

Closing this, since there has been a docker build for a while. If there is still an issue with the docker images, feel free to reopen!