modoboa-installer autodiscover/autoconfig errors and tunnings

Hello!

I tried to summarize infos and problems with autodiscover/autoconfig features.

Modoboa server is installed with Modoboa-installer on Debian Stretch.

In DNS, I have:

autodiscover.domain.tld. CNAME mail.domain.tld.
autoconfig.domain.tld. CNAME mail.domain.tld.
_autodiscover._tcp.domain.tld SRV 1 1 443  mail.domain.tld.
_imaps._tcp.domain.tld SRV 1 1 993  mail.domain.tld.
_pop3s._tcp.domain.tld SRV 10 1 995 mail.domain.tld.

At output, autodiscover is not working from outside request.

If I run this command on the server /srv/automx/env/bin/automx-test [email protected], its working for autoconfig and autodiscover.

But in direct from the navigator with https://autodiscover.domain.tld/autodiscover/autodiscover.xml, I got an 500 error.

mail.domain.tld [pid: 17026|app: 0|req: 12/13] 82.67.159.142 () {44 vars in 860 bytes} [Fri Jun 29 12:05:25 2018] GET /autodiscover/autodiscover.xml => generated 0 bytes in 2 msecs (HTTP/2.0 500) 2 headers in 82 bytes (1 switches on core 0)

autoconfig is working fine.

http://autoconfig.domain.tld/mail/[email protected] works fine with automx-test and with a navigator.

Two issues seems related: #151, #174

More on this.

Working session with automx-test:

2018-06-30 10:22:30,954 DEBUG: DOCUMENT_ROOT: /srv/automx/instance
2018-06-30 10:22:30,954 DEBUG: CONTENT_TYPE: application/x-www-form-urlencoded
2018-06-30 10:22:30,955 DEBUG: wsgi.input: <uwsgi._Input object at 0x7f85d1dbd468>
2018-06-30 10:22:30,955 DEBUG: HTTP_HOST: autodiscover.domain.tld
2018-06-30 10:22:30,955 DEBUG: HTTPS: on
2018-06-30 10:22:30,955 DEBUG: wsgi.multithread: False
2018-06-30 10:22:30,955 DEBUG: HTTP_CONTENT_TYPE: application/x-www-form-urlencoded
2018-06-30 10:22:30,955 DEBUG: REQUEST_URI: /mobileconfig
2018-06-30 10:22:30,955 DEBUG: HTTP_ACCEPT: */*
2018-06-30 10:22:30,956 DEBUG: wsgi.version: (1, 0)
2018-06-30 10:22:30,956 DEBUG: wsgi.run_once: False
2018-06-30 10:22:30,956 DEBUG: wsgi.errors: <open file 'wsgi_errors', mode 'w' at 0x7f85d1da6d20>
2018-06-30 10:22:30,956 DEBUG: REMOTE_PORT: 57682
2018-06-30 10:22:30,956 DEBUG: REQUEST_SCHEME: https
2018-06-30 10:22:30,956 DEBUG: UWSGI_APPID: autodiscover.domain.tld|
2018-06-30 10:22:30,956 DEBUG: uwsgi.version: 2.0.14-debian
2018-06-30 10:22:30,957 DEBUG: HTTP_CONTENT_LENGTH: 67
2018-06-30 10:22:30,957 DEBUG: wsgi.file_wrapper: <built-in function uwsgi_sendfile>
2018-06-30 10:22:30,957 DEBUG: HTTP_ACCEPT_ENCODING: identity
2018-06-30 10:22:30,957 DEBUG: --------------- END environ ---------------
2018-06-30 10:22:30,957 DEBUG: Request POST (raw)

Crashed session with direct call from navigator:

2018-06-30 10:23:39,266 DEBUG: DOCUMENT_ROOT: /srv/automx/instance
2018-06-30 10:23:39,266 DEBUG: wsgi.input: <uwsgi._Input object at 0x7f85d1dbd468>
2018-06-30 10:23:39,266 DEBUG: HTTP_DNT: 1
2018-06-30 10:23:39,266 DEBUG: HTTP_HOST: autodiscover.domain.tld
2018-06-30 10:23:39,266 DEBUG: HTTPS: on
2018-06-30 10:23:39,266 DEBUG: wsgi.multithread: False
2018-06-30 10:23:39,266 DEBUG: REQUEST_URI: /autodiscover/autodiscover.xml
2018-06-30 10:23:39,267 DEBUG: HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
2018-06-30 10:23:39,267 DEBUG: wsgi.version: (1, 0)
2018-06-30 10:23:39,267 DEBUG: wsgi.run_once: False
2018-06-30 10:23:39,267 DEBUG: wsgi.errors: <open file 'wsgi_errors', mode 'w' at 0x7f85d1da6d20>
2018-06-30 10:23:39,267 DEBUG: REMOTE_PORT: 53541
2018-06-30 10:23:39,267 DEBUG: HTTP_ACCEPT_LANGUAGE: fr-fr
2018-06-30 10:23:39,267 DEBUG: REQUEST_SCHEME: https
2018-06-30 10:23:39,268 DEBUG: UWSGI_APPID: autodiscover.domain.tld|
2018-06-30 10:23:39,268 DEBUG: uwsgi.version: 2.0.14-debian
2018-06-30 10:23:39,268 DEBUG: CONTENT_TYPE:
2018-06-30 10:23:39,268 DEBUG: wsgi.file_wrapper: <built-in function uwsgi_sendfile>
2018-06-30 10:23:39,268 DEBUG: HTTP_ACCEPT_ENCODING: br, gzip, deflate
2018-06-30 10:23:39,269 DEBUG: --------------- END environ ---------------
autodiscover.domain.tld [pid: 25376|app: 0|req: 50/60] xx.xx.xx.xx () {44 vars in 770 bytes} [Sat Jun 30 10:23:39 2018] GET /autodiscover/autodiscover.xml => generated 0 bytes in 8 msecs (HTTP/1.1 500) 2 headers in 82 bytes (2 switches on core 0)

I made these modfications to nginx vhosts:

In /etc/nginx/sites-available/autoconfig.domain.tld.conf add this block:

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name autodiscover.domain.tld;
    root /srv/automx/instance;

    ssl_certificate /etc/letsencrypt/live/mail.domain.tld/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/mail.domain.tld/privkey.pem; # managed by Certbot
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:ECDHE-RSA-AES128-GCM-SHA256:AES256+EECDH:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_verify_depth 3;
    ssl_dhparam /etc/nginx/dhparam.pem;

    client_max_body_size 10M;

    access_log /var/log/nginx/autodiscover.domain.tld-access.log;
    error_log /var/log/nginx/autodiscover.domain.tld-error.log;

    location ~* ^/autodiscover/autodiscover.xml {
        include uwsgi_params;
        uwsgi_pass automx;
    }

    location /mail/config-v1.1.xml {
        include uwsgi_params;
        uwsgi_pass automx;
    }

    location /mobileconfig {
        include uwsgi_params;
        uwsgi_pass automx;
    }
}

In /etc/nginx/sites-available/maildomain.tld.conf change:

location /autodiscover/autodiscover.xml {

with (take care of lower/upper case in URL):

location ~* ^/autodiscover/autodiscover.xml

Result.

If you use these tools, it works:

https://testconnectivity.microsoft.com (click on Outlook Autodiscover) https://www.mailenable.com/Tools/AutoDiscover/validate.asp

If I try direct with a navigator, it doesn't work.

Currently, autodiscover doesn't work with Outlook Mac, Apple Mail and iPhone Mail as direct call using https produce a 500 error.

Jul 01 '18 06:07 stefaweb

Found this with nginx debug.

Something generate a Resource temporarily unavailable.

With navigator (tried with Safari, Firefox and Chrome): https://autodiscover.domain.tld/autodiscover/autodiscover.xml In /var/log/nginx/autodiscover.domain.tld-error.log.

2018/07/01 11:41:31 [debug] 3347#3347: *1 http finalize request: -4, "/autodiscover/autodiscover.xml?" a:1, c:2
2018/07/01 11:41:31 [debug] 3347#3347: *1 http request count:2 blk:0
2018/07/01 11:41:31 [debug] 3347#3347: *1 http run request: "/autodiscover/autodiscover.xml?"
2018/07/01 11:41:31 [debug] 3347#3347: *1 http upstream check client, write event:1, "/autodiscover/autodiscover.xml"
2018/07/01 11:41:31 [debug] 3347#3347: *1 http upstream recv(): -1 (11: Resource temporarily unavailable)
2018/07/01 11:41:31 [debug] 3347#3347: *1 http upstream request: "/autodiscover/autodiscover.xml?"
2018/07/01 11:41:31 [debug] 3347#3347: *1 http upstream dummy handler
2018/07/01 11:41:31 [debug] 3347#3347: *1 http upstream request: "/autodiscover/autodiscover.xml?"
2018/07/01 11:41:31 [debug] 3347#3347: *1 http upstream process header
2018/07/01 11:41:31 [debug] 3347#3347: *1 malloc: 000055E56627CF00:4096
2018/07/01 11:41:31 [debug] 3347#3347: *1 recv: fd:22 82 of 4096
2018/07/01 11:41:31 [debug] 3347#3347: *1 http uwsgi status 500 "500 Internal Server Error"
2018/07/01 11:41:31 [debug] 3347#3347: *1 http uwsgi header: "Content-Type: text/html"
2018/07/01 11:41:31 [debug] 3347#3347: *1 http uwsgi header: "Content-Length: 0"
2018/07/01 11:41:31 [debug] 3347#3347: *1 http uwsgi header done
2018/07/01 11:41:31 [debug] 3347#3347: *1 xslt filter header
2018/07/01 11:41:31 [debug] 3347#3347: *1 HTTP/1.1 500 Internal Server Error

With automx-test: In /var/log/nginx/autodiscover.domain.tld-error.log.

2018/07/01 11:42:00 [debug] 3348#3348: *5 http finalize request: -4, "/autodiscover/autodiscover.xml?" a:1, c:2
2018/07/01 11:42:00 [debug] 3348#3348: *5 http request count:2 blk:0
2018/07/01 11:42:00 [debug] 3348#3348: *5 post event 000055E5662BDEC0
2018/07/01 11:42:00 [debug] 3348#3348: *5 delete posted event 000055E5662BDEC0
2018/07/01 11:42:00 [debug] 3348#3348: *5 http run request: "/autodiscover/autodiscover.xml?"
2018/07/01 11:42:00 [debug] 3348#3348: *5 http read client request body
2018/07/01 11:42:00 [debug] 3348#3348: *5 SSL_read: 373
2018/07/01 11:42:00 [debug] 3348#3348: *5 http client request body recv 373
2018/07/01 11:42:00 [debug] 3348#3348: *5 http body new buf t:1 f:0 000055E566271760, pos 000055E566271760, size: 373 file: 0, size: 0
2018/07/01 11:42:00 [debug] 3348#3348: *5 http client request body rest 0
2018/07/01 11:42:00 [debug] 3348#3348: *5 event timer del: 18: 1530438180026
2018/07/01 11:42:00 [debug] 3348#3348: *5 http init upstream, client timer: 0
2018/07/01 11:42:00 [debug] 3348#3348: *5 epoll add event: fd:18 op:3 ev:80002005
2018/07/01 11:42:00 [debug] 3348#3348: *5 posix_memalign: 000055E56627BEB0:512 @16
2018/07/01 11:42:00 [debug] 3348#3348: *5 http script copy: "QUERY_STRING"

With https://www.mailenable.com/Tools/AutoDiscover/validate.asp. In /var/log/nginx/autodiscover.domain.tld-error.log.

2018/07/01 14:27:26 [debug] 3348#3348: *24 http finalize request: -4, "/AutoDiscover/AutoDiscover.xml?" a:1, c:2
2018/07/01 14:27:26 [debug] 3348#3348: *24 http request count:2 blk:0
2018/07/01 14:27:26 [debug] 3348#3348: *24 post event 000055E5662BDEC0
2018/07/01 14:27:26 [debug] 3348#3348: *24 delete posted event 000055E5662BDEC0
2018/07/01 14:27:26 [debug] 3348#3348: *24 http run request: "/AutoDiscover/AutoDiscover.xml?"
2018/07/01 14:27:26 [debug] 3348#3348: *24 http read client request body
2018/07/01 14:27:26 [debug] 3348#3348: *24 SSL_read: 1
2018/07/01 14:27:26 [debug] 3348#3348: *24 SSL_read: 377
2018/07/01 14:27:26 [debug] 3348#3348: *24 http client request body recv 378
2018/07/01 14:27:26 [debug] 3348#3348: *24 http body new buf t:1 f:0 000055E5661F8EF0, pos 000055E5661F8EF0, size: 378 file: 0, size: 0
2018/07/01 14:27:26 [debug] 3348#3348: *24 http client request body rest 0
2018/07/01 14:27:26 [debug] 3348#3348: *24 event timer del: 18: 1530448106482
2018/07/01 14:27:26 [debug] 3348#3348: *24 http init upstream, client timer: 0
2018/07/01 14:27:26 [debug] 3348#3348: *24 epoll add event: fd:18 op:3 ev:80002005
2018/07/01 14:27:26 [debug] 3348#3348: *24 posix_memalign: 000055E5662756F0:512 @16
2018/07/01 14:27:26 [debug] 3348#3348: *24 http script copy: "QUERY_STRING"

Jul 01 '18 09:07 stefaweb

More test.

new block in autoconfig.domain.tld.conf no needed.
we just need to mod location for autodiscover (location ~* ^/autodiscover/autodiscover.xml) to catch lower/upercase.

Still trying to have the right POST in my RestClient but autodiscover works with Windows.

PR #224

Jul 02 '18 09:07 stefaweb

I'm running into a similar issue, currently researching.

Oct 02 '18 17:10 nickdbush

@stefaweb @nickdbush Any news about this issue? Unfortunately, I don't have any mac device to reproduce it...

Nov 28 '18 08:11 tonioo

Issue #253 should interest you guys.

Nov 28 '18 09:11 tonioo

Hi, any progress? I have the same problem accessing http://autoconfig with a 500 error returned. Using https returns a 400 Bad Request: seems that the subdomain autoconfig is not configured to receive its own ssl certificate from LetsEncrypt and meanwhile nginx doesn't redirect to mail.whatsoever (as suggested in this other issue: https://github.com/modoboa/modoboa-installer/issues/222 ) which i tried and anyway responds with a "Page doesn't exists", actually without errors.

Jul 13 '19 12:07 gianks

automx is not installed after auto installer is used. Tested today with ubuntu 18.04 Maybe you should replace automx by z-push available on z-push.org Maybe modoboa will rock again if you do so

Nov 07 '19 22:11 horvan

automx was installed with everything else for me ok (1.14, RasPi Stretch), but only autoconfigure.example.com returns data to requests while a GET request for autodiscover.example.com returns a 500 internal server error.

Running the test from https://testconnectivity.microsoft.com/ returns a positive result when choosing the 'Outlook Autodiscover' option after making some changes to the setup, namely;

moved all the autodiscover and mobileconfig settings from the mail.example.com site config file to the autoconfig.example.com site config in Nginx
created a location in both the Nginx configs to route /.well-know/acme-challenge

    location /.well-known/acme-challenge {
        root /var/www/html;
    }

ran certbot to create a certificate for autodiscover.example.com and autoconfig.example.com in addition to the existing mail.example.com

sudo certbot certonly \
    --webroot -n --agree-tos --force-renewal \
    -w /var/www/html \
    --email [email protected] \
	-d mail.example.com \
	-d autodiscover.example.com \ 
        -d autoconfig.example.com

move autoconfig to SSL in Nginx (copy the method from the mail.example.com config)
changed the instance for automx from 'modoboa' to 'automx' under autodiscover

Nov 11 '19 13:11 dbryar

modoboa-installer modoboa-installer copied to clipboard

autodiscover/autoconfig errors and tunnings

modoboa-installer
modoboa-installer copied to clipboard