[BUG] Insecure option and/or custom CA not working

[jakommo]

Describe the bug

I already checked https://github.com/yllibed/Zigbee2MqttAssistant/issues/81 and https://github.com/yllibed/Zigbee2MqttAssistant/issues/251 but I just can't get it to work with the insecure setting and/or my own CA.

Using -e "Z2MA_SETTINGS__MQTTSECURE=insecure" does not seem to have any effect, it keeps failing with invalid cert.

I wonder if there is something off in general, as it does not even work when adding my root CA cert the image. Dockerfile:

FROM carldebilly/zigbee2mqttassistant:latest

ADD foo.crt /usr/local/share/ca-certificates/foo.crt
RUN chmod 644 /usr/local/share/ca-certificates/foo.crt && update-ca-certificates

RUN apk update && apk add openssl curl # (only needed for debug)

Docker command:

sudo docker run --rm  -p 8880:80 -e "Z2MA_SETTINGS__MQTTSERVER=mqtt.example.lan" -e "Z2MA_SETTINGS__MQTTUSERNAME=$USER" -e "Z2MA_SETTINGS__MQTTPASSWORD=${PW}" -e "Z2MA_SETTINGS__MQTTPORT=8883" -e "Z2MA_SETTINGS__MQTTSECURE=insecure" --name zigbee2mqttassistant $USER/zigbee2mqttassistant

Still results in:

warn: Zigbee2MqttAssistant.Services.MqttConnectionService[0]
      Unable to connect to MQTT server mqtt.example.lan.
MQTTnet.Exceptions.MqttCommunicationException: The remote certificate is invalid according to the validation procedure.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.

Logging into the container and testing if the installed root CA file works:

 sudo docker exec -ti zigbee2mqttassistant  /bin/sh
/app # openssl s_client -connect mqtt.example.lan:8883
CONNECTED(00000003)
....
    Verify return code: 0 (ok)

Checking the result without the root CA file seems to prove that installing the CA file worked, because now openssl says it's 19 (self signed certificate in certificate chain), while it was 0 (ok) before.

/app # rm  /usr/local/share/ca-certificates/foo.crt 
/app # update-ca-certificates 
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
/app # openssl s_client -connect mqtt.example.lan:8883
CONNECTED(00000003)
...
    Verify return code: 19 (self signed certificate in certificate chain)

Any ideas why neither setting insecure nor adding the root CA cert works?

@carldebilly did you get a chance to test/repro this as noted in https://github.com/yllibed/Zigbee2MqttAssistant/issues/81#issuecomment-564195197. There are several users that indicate that the insecure setting isn't working for them.

Steps I took to try resolve the problem

Tried with latest (0.3.164) and with dev . Same behavior

Installation

• Version of Zigbee2Mqtt: .
• Version of Zigbee2MqttAssistant: 0.3.164
• Installation type (HASS.IO, Docker...): Docker
• Docker version + CPU type (amd64, arm arm64...): amd64

Pertinent logs

Starting Zigbee2MqttAssistant v0.3.164+Branch.master.Sha.e20042d5dde758b6f55f222bb8e6a345398e6815...
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
      Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.
warn: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
      No XML encryptor configured. Key {f736fc1e-aad0-4862-b889-2acd086affa6} may be persisted to storage in unencrypted form.
Hosting environment: Production
Content root path: /app
Now listening on: http://[::]:80
Application started. Press Ctrl+C to shut down.
warn: Zigbee2MqttAssistant.Services.MqttConnectionService[0]
      Unable to connect to MQTT server mqtt.example.lan.
MQTTnet.Exceptions.MqttCommunicationException: The remote certificate is invalid according to the validation procedure.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
   at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Security.SslStream.ThrowIfExceptional()
   at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result)
   at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
   at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__64_2(IAsyncResult iar)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at MQTTnet.Implementations.MqttTcpChannel.ConnectAsync(CancellationToken cancellationToken)
   at MQTTnet.Internal.MqttTaskTimeout.WaitAsync(Func`2 action, TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Adapter.MqttChannelAdapter.ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at MQTTnet.Adapter.MqttChannelAdapter.WrapException(Exception exception)
   at MQTTnet.Adapter.MqttChannelAdapter.ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Extensions.ManagedClient.ManagedMqttClient.ReconnectIfRequiredAsync()

[Jeoffreybauvin]

I'm having the same issue here :(.