matrix-docker-ansible-deploy icon indicating copy to clipboard operation
matrix-docker-ansible-deploy copied to clipboard

Published 20 hours ago • verified publisher icon spantaleev

Trouble with intregation with HAProxy GUI via Pfsense

Open RedMageKnight opened this issue 2 years ago • 0 comments

Playbook Configuration:

My vars.yml file looks like this:

---
# The bare domain name which represents your Matrix identity.
# Matrix user ids for your server will be of the form (`@user:<matrix-domain>`).
#
# Note: this playbook does not touch the server referenced here.
# Installation happens on another server ("matrix.<matrix-domain>").
#
# If you've deployed using the wrong domain, you'll have to run the Uninstalling step,
# because you can't change the Domain after deployment.
#
# Example value: example.com
matrix_domain: <root domain>

# The Matrix homeserver software to install.
# See:
#  - `roles/custom/matrix-base/defaults/main.yml` for valid options
# - the `docs/configuring-playbook-IMPLEMENTATION_NAME.md` documentation page, if one is available for your implementation choice
matrix_homeserver_implementation: synapse

# A secret used as a base, for generating various other secrets.
# You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
matrix_homeserver_generic_secret_key: <password>

# This is something which is provided to Let's Encrypt when retrieving SSL certificates for domains.
#
# In case SSL renewal fails at some point, you'll also get an email notification there.
#
# If you decide to use another method for managing SSL certificates (different than the default Let's Encrypt),
# you won't be required to define this variable (see `docs/configuring-playbook-ssl-certificates.md`).
#
# Example value: [email protected]
matrix_ssl_lets_encrypt_support_email: ''

# A Postgres password to use for the superuser Postgres user (called `matrix` by default).
#
# The playbook creates additional Postgres users and databases (one for each enabled service)
# using this superuser account.
matrix_postgres_connection_password: <password>

matrix_jitsi_enabled: true

# Run `bash inventory/scripts/jitsi-generate-passwords.sh` to generate these passwords,
# or define your own strong passwords manually.
matrix_jitsi_jicofo_auth_password: <password>
matrix_jitsi_jvb_auth_password: <password>
matrix_jitsi_jibri_recorder_password: <password>
matrix_jitsi_jibri_xmpp_password: <password>

matrix_jitsi_enable_auth: true
matrix_jitsi_enable_guests: true
matrix_jitsi_prosody_auth_internal_accounts:
  - username: <username>
    password: <password>
  - username: <usermame>
    password: <password>
	
matrix_jitsi_web_custom_config_extension: |
  config.enableLayerSuspension = true;

  config.disableAudioLevels = true;

  // Limit the number of video feeds forwarded to each client
  config.channelLastN = 4;

matrix_jitsi_web_config_resolution_width_ideal_and_max: 480
matrix_jitsi_web_config_resolution_height_ideal_and_max: 240

matrix_etherpad_enabled: true

# Uncomment below if you'd like to install Etherpad on the Dimension domain (not recommended)
# matrix_etherpad_mode: dimension

# Uncomment below to enable the admin web UI
matrix_etherpad_admin_username: <username>
matrix_etherpad_admin_password: <password>

matrix_client_element_themes_enabled: true

matrix_dynamic_dns_enabled: true

matrix_dynamic_dns_domain_configurations:
  - provider: api.cloudflare.com/client/v4
    protocol: cloudflare
    username: <username>
    password: <password>
    domain: <domain>

# Do not retrieve SSL certificates. This shall be managed by another webserver or other means.
matrix_ssl_retrieval_method: none

matrix_nginx_proxy_enabled: false
# Do not try to serve HTTPS, since we have no SSL certificates.
# Disabling this also means services will be served on the HTTP port
# (`matrix_nginx_proxy_container_http_host_bind_port`).
matrix_nginx_proxy_https_enabled: false

# Do not listen for HTTP on port 80 globally (default), listen on the loopback interface.
# If you'd like, you can make it use the local network as well and reverse-proxy from another local machine.
matrix_nginx_proxy_container_http_host_bind_port: '127.0.0.1:81'

# Likewise, expose the Matrix Federation port on the loopback interface.
# Since `matrix_nginx_proxy_https_enabled` is set to `false`, this federation port will serve HTTP traffic.
# If you'd like, you can make it use the local network as well and reverse-proxy from another local machine.
#
# You'd most likely need to expose it publicly on port 8448 (8449 was chosen for the local port to prevent overlap).
matrix_nginx_proxy_container_federation_host_bind_port: '127.0.0.1:8449'

# Coturn relies on SSL certificates that have already been obtained.
# Since we don't obtain any certificates (`matrix_ssl_retrieval_method: none` above), it won't work by default.
# An alternative is to tweak some of: `matrix_coturn_tls_enabled`, `matrix_coturn_tls_cert_path` and `matrix_coturn_tls_key_path`.
matrix_coturn_enabled: false

# Trust the reverse proxy to send the correct `X-Forwarded-Proto` header as it is handling the SSL connection.
matrix_nginx_proxy_trust_forwarded_proto: true

# Trust and use the other reverse proxy's `X-Forwarded-For` header.
matrix_nginx_proxy_x_forwarded_for: '$proxy_add_x_forwarded_for'

matrix_mailer_sender_address: <email>
matrix_mailer_relay_use: true
matrix_mailer_relay_host_name: <host>
matrix_mailer_relay_host_port: 587
matrix_mailer_relay_auth: true
matrix_mailer_relay_auth_username: <username>
matrix_mailer_relay_auth_password: <password>

matrix_synapse_admin_enabled: true

matrix_mautrix_discord_enabled: true

matrix_synapse_ext_password_provider_shared_secret_auth_enabled: true
matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret: <password>
matrix_synapse_password_config_localdb_enabled: false

#matrix_mautrix_hangouts_enabled: true

matrix_mautrix_signal_enabled: true

matrix_mx_puppet_steam_enabled: true

matrix_bot_postmoogle_enabled: true

# Uncomment and adjust this part if you'd like to use a username different than the default
# matrix_bot_postmoogle_login: postmoogle

# Generate a strong password here. Consider generating it with `pwgen -s 64 1`
matrix_bot_postmoogle_password: <password>

matrix_bot_matrix_reminder_bot_enabled: true

# Uncomment and adjust this part if you'd like to use a username different than the default
# matrix_bot_matrix_reminder_bot_matrix_user_id_localpart: bot.matrix-reminder-bot

# Generate a strong password here. Consider generating it with `pwgen -s 64 1`
matrix_bot_matrix_reminder_bot_matrix_user_password: <password>

# Adjust this to your timezone
matrix_bot_matrix_reminder_bot_reminders_timezone: America/New_York

Matrix Server:

  • OS: Ubuntu 22.04.1 LTS
  • Architecture amd64

Problem description:

Hello, I've been trying to take a crack at this here and I'm having an issue getting my proxy to point at the services, after days of looking at it and trying different things and scouring the docs I haven't had any luck. I'm currently trying to run it with the instructions found here in the examples doc for HAProxy in which the built-in NGINX is disabled and I'm running NGINX in a different container on the same server via these instructions: https://github.com/spantaleev/matrix-docker-ansible-deploy/tree/master/examples/haproxy

All of the services appear to be running fine including Synapse, Element, some of the extra bridges I wanted to set up - no issues after running the commands to setup and start everything, but when I try to navigate, I get a 503 response from my proxy, and when I run the test found here on the server after everything is up and running (https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/docs/maintenance-checking-services.md), I get the following error:

TASK [custom/matrix-synapse : Check Matrix Client API] ************************************************************************************************************************************************************************************************************
fatal: [matrix.mydomain.com -> 127.0.0.1]: FAILED! => changed=false 
  cache_control: no-cache
  connection: close
  content_type: text/html
  elapsed: 3
  msg: 'Status code was 503 and not [200]: HTTP Error 503: Service Unavailable'
  redirected: false
  status: 503
  url: https://matrix.mydomain.com/_matrix/client/versions
...ignoring

TASK [custom/matrix-synapse : Fail if Matrix Client API not working] **********************************************************************************************************************************************************************************************
fatal: [matrix.mydomain.com -> 127.0.0.1]: FAILED! => changed=false 
  msg: 'Failed checking Matrix Client API is up at `matrix.mydomain.com` (checked endpoint: `https://matrix.mydomain.com/_matrix/client/versions`). Is Synapse running? Is port 443 open in your firewall? Full error: {''redirected'': False, ''url'': ''https://matrix.mydomain.com/_matrix/client/versions'', ''status'': 503, ''cache_control'': ''no-cache'', ''connection'': ''close'', ''content_type'': ''text/html'', ''elapsed'': 3, ''changed'': False, ''failed'': True, ''msg'': ''Status code was 503 and not [200]: HTTP Error 503: Service Unavailable''}'

PLAY RECAP ********************************************************************************************************************************************************************************************************************************************************
matrix.mydomain.com       : ok=20   changed=0    unreachable=0    failed=1    skipped=144  rescued=0    ignored=1

Synapse does show as running: a22ccbc9be32 matrixdotorg/synapse:v1.71.0 "/start.py run -m sy…" 6 minutes ago Up 6 minutes (healthy) 127.0.0.1:8008->8008/tcp, 8009/tcp, 127.0.0.1:8048->8048/tcp, 8448/tcp matrix-synapse And I have other services being directed through port 443 on my proxy that work just fine, so I'm guessing that there's something going on with how I have the proxy configured to route the traffic, but I'm not sure what I'm missing. Below is my current HAProxy config:

# Automaticaly generated, dont edit manually.
# Generated on: 2022-12-15 16:32
global
	maxconn			10000
	log			/var/run/log	local0	info
	stats socket /tmp/haproxy.socket level admin  expose-fd listeners
	uid			80
	gid			80
	nbproc			1
	nbthread			1
	hard-stop-after		15m
	chroot				/tmp/haproxy_chroot
	daemon
	tune.ssl.default-dh-param	2048
	server-state-file /tmp/haproxy_server_state

listen HAProxyLocalStats
	bind 127.0.0.1:2200 name localstats
	mode http
	stats enable
	stats admin if TRUE
	stats show-legends
	stats uri /haproxy/haproxy_stats.php?haproxystats=1
	timeout client 5000
	timeout connect 5000
	timeout server 5000

frontend matrix-federation
	bind			<public IP>:8448 name <public IP>:8448   ssl crt-list /var/etc/haproxy/matrix-federation.crt_list  
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		30000
	acl			aclcrt_matrix-federation	var(txn.txnhost) -m reg -i ^([^\.]*)\.mydomain\.com(:([0-9]){1,5})?$
	acl			aclcrt_matrix-federation	var(txn.txnhost) -m reg -i ^mydomain\.com(:([0-9]){1,5})?$
	http-request set-var(txn.txnhost) hdr(host)
	use_backend Synapse_ipvANY  if   aclcrt_matrix-federation

frontend matrix-federation-LAN
	bind			192.168.1.1:8448 name 192.168.1.1:8448   ssl crt-list /var/etc/haproxy/matrix-federation-LAN.crt_list  
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		30000
	acl			aclcrt_matrix-federation-LAN	var(txn.txnhost) -m reg -i ^([^\.]*)\.mydomain\.com(:([0-9]){1,5})?$
	acl			aclcrt_matrix-federation-LAN	var(txn.txnhost) -m reg -i ^mydomain\.com(:([0-9]){1,5})?$
	http-request set-var(txn.txnhost) hdr(host)
	use_backend Synapse_ipvANY  if   aclcrt_matrix-federation-LAN

frontend HTTP-to-HTTPS
	bind			<public IP>:80 name <public IP>:80   
	mode			http
	log			global
	option			http-keep-alive
	timeout client		30000
	http-request redirect scheme https 

frontend HTTP-to-HTTPS-LAN
	bind			192.168.1.1:80 name 192.168.1.1:80   
	mode			http
	log			global
	option			http-keep-alive
	timeout client		30000
	http-request redirect scheme https 

frontend SSL-Frontend
	bind			<public IP>:443 name <public IP>:443   ssl crt-list /var/etc/haproxy/SSL-Frontend.crt_list  
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		305000
	acl			matrix_domain	var(txn.txnhost) -m str -i matrix.mydomain.com
	acl			static_files	var(txn.txnpath) -m beg -i /.well-known/matrix
	acl			matrix_identity	var(txn.txnpath) -m beg -i /_matrix/identity
	acl			matrix_search	var(txn.txnpath) -m beg -i /_matrix/client/r0/user_directory/search
	acl			matrix_path	var(txn.txnpath) -m beg -i /_matrix
	acl			synapse_admin	var(txn.txnpath) -m beg -i /_synapse/admin
	acl			element_domain	var(txn.txnhost) -m str -i element.mydomain.com
	acl			aclcrt_SSL-Frontend	var(txn.txnhost) -m reg -i ^([^\.]*)\.mydomain\.com(:([0-9]){1,5})?$
	acl			aclcrt_SSL-Frontend	var(txn.txnhost) -m reg -i ^mydomain\.com(:([0-9]){1,5})?$
	http-request set-var(txn.txnhost) hdr(host)
	http-request set-var(txn.txnpath) path
	use_backend nginx-static_ipvANY  if  static_files aclcrt_SSL-Frontend
	use_backend Matrix-Supporting_ipvANY  if  matrix_identity matrix_search aclcrt_SSL-Frontend
	use_backend Matrix-Main_ipvANY  if  matrix_path synapse_admin aclcrt_SSL-Frontend
	use_backend Element_ipvANY  if  element_domain aclcrt_SSL-Frontend
	use_backend Matrix-Main_ipvANY  if  matrix_domain aclcrt_SSL-Frontend

frontend SSL-Frontend-LAN
	bind			192.168.1.1:443 name 192.168.1.1:443   ssl crt-list /var/etc/haproxy/SSL-Frontend-LAN.crt_list  
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		305000
	acl			matrix_domain	var(txn.txnhost) -m str -i matrix.mydomain.com
	acl			static_files	var(txn.txnpath) -m beg -i /.well-known/matrix
	acl			matrix_identity	var(txn.txnpath) -m beg -i /_matrix/identity
	acl			matrix_search	var(txn.txnpath) -m beg -i /_matrix/client/r0/user_directory/search
	acl			matrix_path	var(txn.txnpath) -m beg -i /_matrix
	acl			synapse_admin	var(txn.txnpath) -m beg -i /_synapse/admin
	acl			element_domain	var(txn.txnhost) -m str -i element.mydomain.com
	acl			aclcrt_SSL-Frontend-LAN	var(txn.txnhost) -m reg -i ^([^\.]*)\.mydomain\.com(:([0-9]){1,5})?$
	acl			aclcrt_SSL-Frontend-LAN	var(txn.txnhost) -m reg -i ^mydomain\.com(:([0-9]){1,5})?$
	http-request set-var(txn.txnhost) hdr(host)
	http-request set-var(txn.txnpath) path
	use_backend nginx-static_ipvANY  if  static_files aclcrt_SSL-Frontend-LAN
	use_backend Matrix-Supporting_ipvANY  if  matrix_identity or matrix_search aclcrt_SSL-Frontend-LAN
	use_backend Matrix-Main_ipvANY  if  matrix_path or synapse_admin aclcrt_SSL-Frontend-LAN
	use_backend Element_ipvANY  if  element_domain aclcrt_SSL-Frontend-LAN
	use_backend Matrix-Main_ipvANY  if  matrix_domain aclcrt_SSL-Frontend-LAN

backend Synapse_ipvANY
	mode			http
	id			120
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	server			synapse 192.168.1.173:8048 id 121    

backend nginx-static_ipvANY
	mode			http
	id			122
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	server			nginx 192.168.1.173:40888 id 123  

backend Matrix-Supporting_ipvANY
	mode			http
	id			114
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	server			matrix-supporting 192.168.1.173:8090 id 115  

backend Matrix-Main_ipvANY
	mode			http
	id			118
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	server			matrix-main 192.168.1.173:8008 id 119  

backend Element_ipvANY
	mode			http
	id			124
	log			global
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk OPTIONS / 
	server			element 192.168.1.173:8765 id 125 check inter 1000

Would anyone be able to offer some assistance with this? I would greatly appreciate it! Thank you!

RedMageKnight avatar Dec 16 '22 15:12 RedMageKnight