docker-postfix icon indicating copy to clipboard operation
docker-postfix copied to clipboard
verified publisher icon bokysan

Settings not applied consistently

Open MrEbbinghaus opened this issue 2 years ago • 8 comments

The default mynetwork setting isn't applied consistently.

Problem is, I can't reliably reproduce it. Sometimes when I run docker compose up it just doesn't get set, most of the time it does. I haven't changed any settings between restarts..

I don't have POSTFIX_mynetwork set and I don't have any config in files, only docker labels.

On startup the logs show the right networks:

INFO  Applying custom postfix setting: mynetworks=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16

But postconf gives me:

postconf | grep "mynetworks ="
mynetworks = 127.0.0.1/32 172.23.0.5/32

There is no setting set in main.cf

# Alternatively, you can specify the mynetworks list by hand, in
# which case Postfix ignores the mynetworks_style setting.
#
# Specify an explicit list of network/netmask patterns, where the
# mask specifies the number of bits in the network part of a host
# address.
#
# You can also specify the absolute pathname of a pattern file instead
# of listing the patterns here. Specify type:table for table-based lookups
# (the value on the table right-hand side is not used).
#
#mynetworks = 168.100.3.0/28, 127.0.0.0/8
#mynetworks = $config_directory/mynetworks
#mynetworks = hash:/etc/postfix/network_table

# ...
myhostname = <redacted>
# mynetworks is missing here. Most of the time it's: mynetworks = 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
myorigin = <redacted>

(And nothing at the end of the file as well.)

MrEbbinghaus avatar Jan 02 '24 22:01 MrEbbinghaus

Well, this is a first one.

Unfortunately, you have not given me a lot to go on:

  • which version of the image are you using
  • which variant (Debian, Alpine...)
  • what's your startup procedure / configuration settings for running the image
  • does this happen with mynetworks only, or are the other settings affected as well
  • do you have any volumes bound
  • ...

So, until I'm able to reproduce the issue, unfortunately, I don't think I can help.

bokysan avatar Jan 03 '24 06:01 bokysan

It's pretty weird and I'm pretty confused about what could cause that behavior. I've looked at the startup scripts in this repo and found nothing on first glance.

But you are right, I could at least have provided more details:

For now I have only seen mynetworks missing. Which I noticed, because Postfix started to reject mail from the local Mailman.

I'm using boky/postfix:latest-alpine (at the time of writing that's c2b9a8a68d73) I use it as a docker compose service with restart: unless-stopped. Port 25 is exposed to the host and the container is part a bridge network for mailman to be able to talk via LMTP.

Config

(Everything configured via the POSTFIX_ environment variables.)

I have two binds:

volumes:
      - ./volumes/mailman-core:/opt/mailman/core:ro
      - /host/keys:/etc/opendkim/keys

/opt/mailman/core contains transport_maps, local_recipient_maps and relay_domains that are auto-generated by Mailman.

Additionally I have set mydomain and myorigin.

DKIM_AUTOGENERATE: "true"
DKIM_SELECTOR: 10-2023

POSTFIX_mydomain: <redacted>
POSTFIX_myorigin: <redacted> # Default: $myhostname localhost.$mydomain localhost

POSTFIX_recipient_delimiter: +
POSTFIX_unknown_local_recipient_reject_code: 550
POSTFIX_owner_request_special: "no"
POSTFIX_transport_maps: regexp:/opt/mailman/core/var/data/postfix_lmtp
POSTFIX_local_recipient_maps: regexp:/opt/mailman/core/var/data/postfix_lmtp
POSTFIX_relay_domains: regexp:/opt/mailman/core/var/data/postfix_domains
# And a bunch of restrictions. (client, helo, sender, recipient)
# They use `permit_mynetworks` which of course fails if mynetworks isn't set.

MrEbbinghaus avatar Jan 09 '24 14:01 MrEbbinghaus

@bokysan I can now reproduce my problem:

$ docker compose down && docker compose up -d

$ docker exec mailman-postfix-1 postconf | grep "mynetworks ="
mynetworks = 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16

$ docker compose restart postfix

$ docker exec mailman-postfix-1 postconf | grep "mynetworks ="
mynetworks = 127.0.0.1/32 172.23.0.2/32

Curiosly the same happens with --force-recreate:

$ docker compose down && docker compose up -d

$ docker exec mailman-postfix-1 postconf | grep "mynetworks ="
mynetworks = 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16

$ docker compose up -d --force-recreate

$ docker exec mailman-postfix-1 postconf | grep "mynetworks ="
mynetworks = 127.0.0.1/32 172.23.0.2/32

(I waited for the startup script to post INFO Applying custom postfix setting: mynetworks=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 after every start.)

MrEbbinghaus avatar Feb 06 '24 12:02 MrEbbinghaus

This is really odd. Can you check the logs from docker compose up -d and then from docker compose restart postfix?

The image is pretty chatty about what's it doing. I just want to ensure to know where this is coming from.

bokysan avatar Feb 06 '24 12:02 bokysan

@bokysan Sure. I'm pretty confused by that. :-) I thought at least with the --force-recreate you have the complete container lifecycle and thus it should be equivalent to down && up.

$ docker compose down
$ docker compose up -d
$ docker compose logs postfix
postfix-1  | ★★★★★ POSTFIX STARTING UP (alpine) ★★★★★
postfix-1  | ‣ NOTE  Setting container timezone to: Europe/Berlin
postfix-1  | ‣ INFO  Using plain log format for rsyslog.
postfix-1  | ‣ NOTE  Emails in the logs will not be anonymized. Set ANONYMIZE_EMAILS to enable this feature.
postfix-1  | ‣ DEBUG Reowning root: /var/spool/postfix/
postfix-1  | ‣ DEBUG Reowning root: /var/spool/postfix/pid/
postfix-1  | ‣ DEBUG Reowning postfix:postdrop /var/spool/postfix/private/
postfix-1  | ‣ DEBUG Reowning postfix:postdrop /var/spool/postfix/public/
postfix-1  | ‣ INFO  Preparing files for Postfix chroot:
postfix-1  | ln: /var/spool/postfix/usr/lib/zoneinfo/: No such file or directory
postfix-1  |         '/var/spool/postfix/usr/lib/zoneinfo/' -> '/etc/localtime'
postfix-1  |         '/etc/localtime' -> '/var/spool/postfix/etc'
postfix-1  |         '/etc/nsswitch.conf' -> '/var/spool/postfix/etc'
postfix-1  |         '/etc/resolv.conf' -> '/var/spool/postfix/etc'
postfix-1  |         '/etc/services' -> '/var/spool/postfix/etc'
postfix-1  |         '/etc/hosts' -> '/var/spool/postfix/etc'
postfix-1  |         '/etc/passwd' -> '/var/spool/postfix/etc'
postfix-1  | ‣ DEBUG No upgrade of hashes needed needed.
postfix-1  | ‣ INFO  Using unlimited message size.
postfix-1  | ‣ INFO  Setting smtp_tls_security_level: may
postfix-1  | ‣ NOTE  Postfix is configured to deliver messages directly (without relaying). Make sure your DNS is setup properly! If unsure, read the docs.
postfix-1  | ‣ INFO  Using default private network list for trusted networks.
postfix-1  | ‣ INFO  Debugging is disabled.
postfix-1  | ‣ INFO  Setting up allowed SENDER domains: YYYYYYYYY XXXXXXXXX ZZZZZZZZZZ
postfix-1  | ‣ INFO  DKIM_AUTOGENERATE set -- will try to auto-generate keys for YYYYYYYYY  XXXXXXXXX ZZZZZZZZZZ
postfix-1  | .
postfix-1  | ‣ INFO  Key for domain YYYYYYYYY already exists in /etc/opendkim/keys/YYYYYYYYY.private. Will not overwrite.
postfix-1  | ‣ INFO  Key for domain XXXXXXXXX already exists in /etc/opendkim/keys/XXXXXXXXX.private. Will not overwrite.
postfix-1  | ‣ INFO  Key for domain ZZZZZZZZZZ already exists in /etc/opendkim/keys/ZZZZZZZZZZ.private. Will not overwrite.
postfix-1  | ‣ NOTE  Configuring OpenDKIM.
postfix-1  |         ...using socket inet:localhost:8891
postfix-1  |         ...for domain YYYYYYYYY (selector: 10-2023)
postfix-1  |         ...for domain XXXXXXXXX (selector: 10-2023)
postfix-1  |         ...for domain ZZZZZZZZZZ (selector: 10-2023)
postfix-1  | ‣ INFO  Applying custom postfix setting: local_recipient_maps=regexp:/opt/mailman/core/var/data/postfix_lmtp
postfix-1  | ‣ INFO  Applying custom postfix setting: message_size_limit=0
postfix-1  | ‣ INFO  Applying custom postfix setting: mydomain=XXXXXXXXX
postfix-1  | ‣ INFO  Applying custom postfix setting: myhostname=XXXXXXXXX
postfix-1  | ‣ INFO  Applying custom postfix setting: mynetworks=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
postfix-1  | ‣ INFO  Applying custom postfix setting: myorigin=XXXXXXXXX
postfix-1  | ‣ INFO  Applying custom postfix setting: owner_request_special=no
postfix-1  | ‣ INFO  Applying custom postfix setting: recipient_delimiter=+
postfix-1  | ‣ INFO  Applying custom postfix setting: relay_domains=regexp:/opt/mailman/core/var/data/postfix_domains
postfix-1  | ‣ INFO  Applying custom postfix setting: smtp_tls_security_level=may
postfix-1  | ‣ INFO  Deleting custom postfix setting: smtpd_client_restrictions
postfix-1  | ‣ INFO  Applying custom postfix setting: smtpd_helo_restrictions=permit_mynetworks,  reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, reject_unknown_helo_hostname,  permit
postfix-1  | ‣ INFO  Applying custom postfix setting: smtpd_recipient_restrictions=permit_mynetworks, reject_unauth_pipelining, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_destination, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net, permit
postfix-1  | ‣ INFO  Applying custom postfix setting: smtpd_sender_restrictions=permit_mynetworks, reject_non_fqdn_sender, reject_unknown_sender_domain, permit
postfix-1  | ‣ INFO  Applying custom postfix setting: transport_maps=regexp:/opt/mailman/core/var/data/postfix_lmtp
postfix-1  | ‣ INFO  Applying custom postfix setting: unknown_local_recipient_reject_code=550
postfix-1  | ‣ NOTE  Starting: rsyslog, crond, postfix
postfix-1  | 2024-02-06 13:50:51,205 INFO Set uid to user 0 succeeded
postfix-1  | 2024-02-06 13:50:51,210 INFO supervisord started with pid 1
postfix-1  | 2024-02-06 13:50:52,227 INFO spawned: 'cron' with pid 213
postfix-1  | 2024-02-06 13:50:52,241 INFO spawned: 'opendkim' with pid 214
postfix-1  | 2024-02-06 13:50:52,244 INFO spawned: 'postfix' with pid 216
postfix-1  | 2024-02-06 13:50:52,253 INFO spawned: 'rsyslog' with pid 218
postfix-1  | 2024-02-06T13:50:52.288128+01:00 INFO    : [origin software="rsyslogd" swVersion="8.2310.0" x-pid="218" x-info="https://www.rsyslog.com"] start
postfix-1  | 2024-02-06 13:50:52,299 INFO success: cron entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
postfix-1  | 2024-02-06 13:50:52,299 INFO success: postfix entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
postfix-1  | 2024-02-06T13:50:52.814202+01:00 INFO    postfix/postfix-script[287]: starting the Postfix mail system
postfix-1  | 2024-02-06T13:50:52.822653+01:00 INFO    postfix/master[288]: daemon started -- version 3.8.4, configuration /etc/postfix
postfix-1  | 2024-02-06 13:50:54,825 INFO success: rsyslog entered RUNNING state, process has stayed up for > than 2 seconds (startsecs)
postfix-1  | 2024-02-06 13:50:57,829 INFO success: opendkim entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)
$ docker exec mailman-postfix-1 postconf | grep "mynetworks ="
mynetworks = 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16

$ docker compose restart postfix
$ docker compose logs postfix
postfix-1  | 2024-02-06 13:53:59,791 WARN received SIGTERM indicating exit request
postfix-1  | 2024-02-06 13:53:59,792 INFO waiting for cron, opendkim, postfix, rsyslog to die
postfix-1  | 2024-02-06 13:53:59,795 INFO stopped: rsyslog (exit status 0)
postfix-1  | 2024-02-06 13:53:59,800 WARN stopped: postfix (terminated by SIGTERM)
postfix-1  | 2024-02-06 13:54:02,805 INFO waiting for cron, opendkim to die
postfix-1  | 2024-02-06 13:54:02,835 INFO stopped: opendkim (exit status 0)
postfix-1  | 2024-02-06 13:54:02,838 WARN stopped: cron (terminated by SIGTERM)
postfix-1  | ★★★★★ POSTFIX STARTING UP (alpine) ★★★★★
postfix-1  | ‣ NOTE  Setting container timezone to: Europe/Berlin
postfix-1  | ‣ INFO  Using plain log format for rsyslog.
postfix-1  | ‣ NOTE  Emails in the logs will not be anonymized. Set ANONYMIZE_EMAILS to enable this feature.
postfix-1  | ‣ DEBUG Reowning root: /var/spool/postfix/
postfix-1  | ‣ DEBUG Reowning root: /var/spool/postfix/pid/
postfix-1  | ‣ DEBUG Reowning postfix:postdrop /var/spool/postfix/private/
postfix-1  | ‣ DEBUG Reowning postfix:postdrop /var/spool/postfix/public/
postfix-1  | ‣ INFO  Preparing files for Postfix chroot:
postfix-1  | ln: /var/spool/postfix/usr/lib/zoneinfo/: No such file or directory
postfix-1  |         '/var/spool/postfix/usr/lib/zoneinfo/' -> '/etc/localtime'
postfix-1  |         '/etc/localtime' -> '/var/spool/postfix/etc'
postfix-1  |         '/etc/nsswitch.conf' -> '/var/spool/postfix/etc'
postfix-1  |         '/etc/resolv.conf' -> '/var/spool/postfix/etc'
postfix-1  |         '/etc/services' -> '/var/spool/postfix/etc'
postfix-1  |         '/etc/hosts' -> '/var/spool/postfix/etc'
postfix-1  |         '/etc/passwd' -> '/var/spool/postfix/etc'
postfix-1  | ‣ DEBUG No upgrade of hashes needed needed.
postfix-1  | ‣ INFO  Using unlimited message size.
postfix-1  | ‣ INFO  Setting smtp_tls_security_level: may
postfix-1  | ‣ NOTE  Postfix is configured to deliver messages directly (without relaying). Make sure your DNS is setup properly! If unsure, read the docs.
postfix-1  | ‣ INFO  Using default private network list for trusted networks.
postfix-1  | ‣ INFO  Debugging is disabled.
postfix-1  | ‣ INFO  Setting up allowed SENDER domains: YYYYYYYYY XXXXXXXXX ZZZZZZZZZZ
postfix-1  | ‣ INFO  DKIM_AUTOGENERATE set -- will try to auto-generate keys for YYYYYYYYY  XXXXXXXXX ZZZZZZZZZZ
postfix-1  | .
postfix-1  | ‣ INFO  Key for domain YYYYYYYYY already exists in /etc/opendkim/keys/YYYYYYYYY.private. Will not overwrite.
postfix-1  | ‣ INFO  Key for domain XXXXXXXXX already exists in /etc/opendkim/keys/XXXXXXXXX.private. Will not overwrite.
postfix-1  | ‣ INFO  Key for domain ZZZZZZZZZZ already exists in /etc/opendkim/keys/ZZZZZZZZZZ.private. Will not overwrite.
postfix-1  | ‣ NOTE  Configuring OpenDKIM.
postfix-1  |         ...using socket inet:localhost:8891
postfix-1  |         ...for domain YYYYYYYYY (selector: 10-2023)
postfix-1  |         ...for domain XXXXXXXXX (selector: 10-2023)
postfix-1  |         ...for domain ZZZZZZZZZZ (selector: 10-2023)
postfix-1  | ‣ INFO  Applying custom postfix setting: local_recipient_maps=regexp:/opt/mailman/core/var/data/postfix_lmtp
postfix-1  | ‣ INFO  Applying custom postfix setting: message_size_limit=0
postfix-1  | ‣ INFO  Applying custom postfix setting: mydomain=XXXXXXXXX
postfix-1  | ‣ INFO  Applying custom postfix setting: myhostname=XXXXXXXXX
postfix-1  | ‣ INFO  Applying custom postfix setting: mynetworks=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
postfix-1  | ‣ INFO  Applying custom postfix setting: myorigin=XXXXXXXXX
postfix-1  | ‣ INFO  Applying custom postfix setting: owner_request_special=no
postfix-1  | ‣ INFO  Applying custom postfix setting: recipient_delimiter=+
postfix-1  | ‣ INFO  Applying custom postfix setting: relay_domains=regexp:/opt/mailman/core/var/data/postfix_domains
postfix-1  | ‣ INFO  Applying custom postfix setting: smtp_tls_security_level=may
postfix-1  | ‣ INFO  Deleting custom postfix setting: smtpd_client_restrictions
postfix-1  | ‣ INFO  Applying custom postfix setting: smtpd_helo_restrictions=permit_mynetworks,  reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, reject_unknown_helo_hostname,  permit
postfix-1  | ‣ INFO  Applying custom postfix setting: smtpd_recipient_restrictions=permit_mynetworks, reject_unauth_pipelining, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_destination, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net, permit
postfix-1  | ‣ INFO  Applying custom postfix setting: smtpd_sender_restrictions=permit_mynetworks, reject_non_fqdn_sender, reject_unknown_sender_domain, permit
postfix-1  | ‣ INFO  Applying custom postfix setting: transport_maps=regexp:/opt/mailman/core/var/data/postfix_lmtp
postfix-1  | ‣ INFO  Applying custom postfix setting: unknown_local_recipient_reject_code=550
postfix-1  | ‣ NOTE  Starting: rsyslog, crond, postfix
postfix-1  | 2024-02-06 13:54:08,936 INFO Set uid to user 0 succeeded
postfix-1  | 2024-02-06 13:54:08,942 INFO supervisord started with pid 1
postfix-1  | 2024-02-06 13:54:09,946 INFO spawned: 'cron' with pid 216
postfix-1  | 2024-02-06 13:54:09,949 INFO spawned: 'opendkim' with pid 217
postfix-1  | 2024-02-06 13:54:09,951 INFO spawned: 'postfix' with pid 218
postfix-1  | 2024-02-06 13:54:09,954 INFO spawned: 'rsyslog' with pid 219
postfix-1  | 2024-02-06T13:54:09.961249+01:00 INFO    : [origin software="rsyslogd" swVersion="8.2310.0" x-pid="219" x-info="https://www.rsyslog.com"] start
postfix-1  | 2024-02-06 13:54:09,961 INFO success: cron entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
postfix-1  | 2024-02-06 13:54:09,962 INFO success: postfix entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
postfix-1  | 2024-02-06T13:54:10.851575+01:00 INFO    postfix/postfix-script[290]: starting the Postfix mail system
postfix-1  | 2024-02-06T13:54:10.859672+01:00 INFO    postfix/master[291]: daemon started -- version 3.8.4, configuration /etc/postfix
postfix-1  | 2024-02-06 13:54:12,862 INFO success: rsyslog entered RUNNING state, process has stayed up for > than 2 seconds (startsecs)
postfix-1  | 2024-02-06 13:54:15,866 INFO success: opendkim entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)
$ docker exec mailman-postfix-1 postconf | grep "mynetworks ="
mynetworks = 127.0.0.1/32 172.27.0.3/32

MrEbbinghaus avatar Feb 06 '24 12:02 MrEbbinghaus

Ok, in both cases it says:

postfix-1  | ‣ INFO  Applying custom postfix setting: mynetworks=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16

The only possibility I see is the if the do_postconf method wrongly deducts that this setting and instead of setting it, it deletes it.

I will try to recreate your issue and see what happens.

bokysan avatar Feb 09 '24 08:02 bokysan

The same actually happens, when I set POSTFIX_mynetworks manually. POSTFIX_mynetworks: 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16

MrEbbinghaus avatar Feb 11 '24 12:02 MrEbbinghaus

Hi @MrEbbinghaus -- so far I believe you're the only person with this problem and I have not been able to replicate the issue anywhere.

Not quite sure where to go from here.

I'd kindly ask you to other provide:

  • a test case which consistently reproduces a problem, or
  • a patch fix

Appreciated, B

bokysan avatar Apr 23 '24 06:04 bokysan