docker-recorder icon indicating copy to clipboard operation
docker-recorder copied to clipboard

Published 20 hours ago • verified publisher icon owntracks

Recorder and mqtt client disconnected and not authorised

Open shadowjig opened this issue 3 years ago • 8 comments

I'm very frustrated trying to get the docker images working. I've stripped down the images and run mqtt separately from the otrecorder. I can get mqtt running and connect an app on my phone to mqtt without issue using the username I setup. However, if I run otrecorder and point it at the same IP address I get an error that the client is not authorised.

Here are the relevant mqtt logs. You can see the ot-recorder* is disconnected and not authorised. Even though it's using the same settings and username/password combo as the "test" client connecting from 10.100.1.11. I redacted the username.

2022-02-15T04:11:02.124726875Z 1644898262: New connection from 10.1.6.15:44465 on port 1883.
2022-02-15T04:11:02.125608165Z 1644898262: Client ot-recorder-897ffc6baf0a-12 disconnected, not authorised.
2022-02-15T04:13:24.268276982Z 1644898404: New connection from 10.100.1.11:44668 on port 1883.
2022-02-15T04:13:24.360620855Z 1644898404: New client connected from 10.100.1.11:44668 as test (p2, c1, k15, u'<username>').
2022-02-15T04:13:27.294632181Z 1644898407: Client test disconnected.
2022-02-15T04:13:36.641410137Z 1644898416: New connection from 10.1.6.15:44512 on port 1883.
2022-02-15T04:13:36.642553900Z 1644898416: Client ot-recorder-897ffc6baf0a-11 disconnected, not authorised.

I can also set the allow_anonymous to false in the mosquitto.conf file and owntracks can connect, but I don't want to leave it that way. It's not a network issue, both clients are connecting via IP and are not on the localhost (same host as mqtt). It's not a username/password issue, both clients are using the same credentials.

The only thing left is how owntracks is configured to connect and send the client info. And I don't see anything in the documentation that requires that type of setup. How can I possible get this working? What's next to review, config and test it?

shadowjig avatar Feb 15 '22 04:02 shadowjig

Have you invoked ot-recorder with -e OTR_USER=user -e OTR_PASS=pass containing credentials for your MQTT broker?

You shouldn't have to allow anonymous clients on your MQTT broker, on the contrary. If you show us a bit more of your configuration maybe we can see what's going on.

jpmens avatar Feb 15 '22 10:02 jpmens

I've tried adding the credentials in the compose files as well as the recorder.conf file and neither work. Below is my current config, docker compose files for mqtt and OTR and the recorder.conf and mosquitto.conf files. This config allows my phone to connect to mqtt but OTR cannot connect (I get the not authorised error in my original post). Also I'm on a large /16 network, so the IP's in my original post will look a little odd. But all the IP's are in the same subnet.

############################################################################
###  Owntracks recorder container
############################################################################

version: '3'

networks:
  docker_ip_network:
    external: true
    
services:
  otrecorder:
    container_name: owntracks_recorder
    image: owntracks/recorder
    restart: unless-stopped
    networks:
      docker_ip_network:
        ipv4_address: <IP for OTR container>
    ports:
      - 8083:8083
    environment:
      - TZ=America/New_York
      - OTR_USER="<redacted>"
      - OTR_PASS="<redacted>"
      - OTR_HOST=<IP for mqtt container>
      - OTR_PORT=1883
    volumes:
      - /volume1/docker/owntracks/config:/config
      - /volume1/docker/owntracks/store:/store
	  
############################################################################
###  mqtt container
############################################################################

version: '3'

networks:
   docker_ip_network:
      external: true
    
services:
   mosquitto:
      container_name: mosquitto
      restart: unless-stopped
      image: eclipse-mosquitto:2-openssl  #needed to use this version because the mosquitto_passwd command does not work in :latest
      volumes:
         - /volume1/docker/mosquitto/data:/mosquitto/data
         - /volume1/docker/mosquitto/config:/mosquitto/config
         - /volume1/docker/mosquitto/log:/mosquitto/log
      networks:
         docker_ip_network:
            ipv4_address: <IP for mqtt container>
      ports:
         - "1883:1883"
         - "8883:8883"
		 
############################################################################
###  mosquitto.conf
############################################################################
listener 1883 0.0.0.0
allow_anonymous false
password_file /mosquitto/config/passwd
use_username_as_clientid false

############################################################################
###  recorder.conf
############################################################################
#(@)ot-recorder.default
#
# Specify global configuration options for the OwnTracks Recorder
# and its associated utilities to override compiled-in defaults.

OTR_HOST="<IP for mqtt container>"
OTR_PORT="1883"
OTR_USER="<redacted>"
OTR_PASS="<redacted>"
OTR_TOPICS = "owntracks/#"

shadowjig avatar Feb 15 '22 15:02 shadowjig

Any idea why this is occurring? It seems like it might be how the owntracks recorder is forming the messages to mqtt. But I don't know how to confirm that via the logs. Is there anything else I can look at to help identify the issue?

shadowjig avatar Feb 23 '22 17:02 shadowjig

Your configuration with OTR_USER and OTR_PASS looks ok to me. Can you show logs of Mosquitto when the recorder tries connecting?

jpmens avatar May 16 '22 06:05 jpmens

Your configuration with OTR_USER and OTR_PASS looks ok to me. Can you show logs of Mosquitto when the recorder tries connecting?

Here are the logs. As I mentioned in my first post. I can get a client on my phone to connect without issue (see last log below). It must be an issue with the configuration within ot-recorder that's causing the issue.

***Logs from mosquitto container 2022-06-12T14:54:32.860810127Z 1655045672: New connection from 10.1.6.15:47299 on port 1883. 2022-06-12T14:54:32.907613603Z 1655045672: Client ot-recorder-0e2f583a8fe1-11 disconnected, not authorised. 2022-06-12T14:54:42.884389900Z 1655045682: New connection from 10.1.6.15:47303 on port 1883. 2022-06-12T14:54:42.885236424Z 1655045682: Client ot-recorder-0e2f583a8fe1-11 disconnected, not authorised. 2022-06-12T14:54:52.937320461Z 1655045692: New connection from 10.1.6.15:47307 on port 1883. 2022-06-12T14:54:52.938191203Z 1655045692: Client ot-recorder-0e2f583a8fe1-11 disconnected, not authorised.

***Logs from ot-recorder 2022-06-12T14:54:32.882647883Z ot-recorder[11]: HTTP listener started on 127.0.0.1:8083 2022-06-12T14:54:32.882955374Z ot-recorder[11]: Using storage at /store with precision 7 2022-06-12T14:54:32.883135110Z ot-recorder[11]: Subscribing to owntracks/# (qos=2) 2022-06-12T14:54:32.883246719Z ot-recorder[11]: Disconnected. Reason: 0x5 [Connection refused: bad username or password] 2022-06-12T14:54:32.883356332Z ot-recorder[11]: MQTT connection: rc=5 [The connection was refused.] (errno=2; No such file or directory). Sleeping...

***Here's the log showing the "op9" phone client (10.1.1.37) is connected by ot-recorder (10.1.6.15) is not connecting. 2022-06-12T15:04:36.021095910Z 1655046276: Client ot-recorder-0e2f583a8fe1-11 disconnected, not authorised. 2022-06-12T15:04:46.074433286Z 1655046286: New connection from 10.1.6.15:47515 on port 1883. 2022-06-12T15:04:46.075302926Z 1655046286: Client ot-recorder-0e2f583a8fe1-11 disconnected, not authorised. 2022-06-12T15:04:47.649744745Z 1655046287: New connection from 10.1.1.37:47232 on port 1883. 2022-06-12T15:04:47.687486594Z 1655046287: New client connected from 10.1.1.37:47232 as op9 (p2, c0, k1200, u'shadowjig'). 2022-06-12T15:04:56.128363581Z 1655046296: New connection from 10.1.6.15:47518 on port 1883. 2022-06-12T15:04:56.129223921Z 1655046296: Client ot-recorder-0e2f583a8fe1-11 disconnected, not authorised.

shadowjig avatar Jun 12 '22 15:06 shadowjig

Have you double-checked the IP address of the MQTT broker the Recorder is trying to connect to? (i.e. do you have other brokers it might be attempting?)

I just don't see it. There must be a disagreement in the credentials.

jpmens avatar Jun 14 '22 11:06 jpmens

What I can offer is to look at your files, if you send them to us at [email protected]:

  • mosquitto.conf
  • /mosquitto/config/passwd
  • recorder.conf

(the second one to verify that the entry you're using actually exists)

jpmens avatar Jun 14 '22 11:06 jpmens

@shadowjig life interrupted me, and I've lost track where we stand here. Have you meanwhile succeeded?

jpmens avatar Aug 04 '22 12:08 jpmens

Closing as stale.

jpmens avatar Sep 07 '22 11:09 jpmens

@jpmens I've gotten back to this again. I updated my docker images. Pulling the latest of both owntracks/recorder and eclipse-mosquitto. The same issues exist now as well. I can connect via a phone (ip 10.1.1.102) but not the owntracks container (ip 10.1.6.16). mqtt is running with ip 10.1.6.17. I have a large logical network so all these hosts are actually on the same logical network. This time I noticed the allow_anonymous true doesn't seem to work now. The username is user and the password is pass.

Below are the logs from mosquitto, mosquitto.conf, recorder.conf, passwd file, and my docker-compose.yml file.

[Logs from mosquitto]
1669854990: mosquitto version 2.0.15 starting
1669854990: Config loaded from /mosquitto/config/mosquitto.conf.
1669854990: Opening ipv4 listen socket on port 1883.
1669854990: mosquitto version 2.0.15 running
1669854991: New connection from 10.1.6.16:55855 on port 1883.
1669854991: Client otrec disconnected, not authorised.
1669855001: New connection from 10.1.6.16:55869 on port 1883.
1669855001: Client otrec disconnected, not authorised.
1669855011: New connection from 10.1.6.16:55881 on port 1883.
1669855011: Client otrec disconnected, not authorised.
1669855021: New connection from 10.1.6.16:55897 on port 1883.
1669855021: Client otrec disconnected, not authorised.
1669855031: New connection from 10.1.6.16:55909 on port 1883.
1669855031: Client otrec disconnected, not authorised.
1669855032: New connection from 10.1.1.102:40344 on port 1883.
1669855032: New client connected from 10.1.1.102:40344 as phone_client (p2, c1, k15, u'user').
1669855038: Client phone_client disconnected.
1669855041: New connection from 10.1.6.16:55921 on port 1883.
1669855041: Client otrec disconnected, not authorised.
1669855041: New connection from 10.1.1.102:40346 on port 1883.
1669855041: New client connected from 10.1.1.102:40346 as phone_client (p2, c1, k15, u'user').

[recorder.conf]
OTR_HOST="10.1.6.17"
OTR_PORT="1883"
OTR_USER="user"
OTR_PASS="pass"
OTR_CLIENTID="otrec"
OTR_TOPICS="owntracks"


[mosquitto.conf]
listener 1883 
socket_domain ipv4
allow_anonymous true
password_file /mosquitto/config/passwd

[passwd]
user:$7$101$sMYtwG5Aro1aMEfi$7zx1WcMO2WijwhOaBzrM4mHUIMf3ipMt8liq8ishV58dLPAhXr1GVuJ+TigjGJcyJv5se0icjwYS/XaMSsJjkw==

[docker-compose.yml]
############################################################################################
###  Owntracks recorder - location tracking app
############################################################################################

version: '3'

networks:
  docker_ip_network:
    external: true
    
services:
  otrecorder:
    image: owntracks/recorder
    networks:
      docker_ip_network:
        ipv4_address: 10.1.6.16
    ports:
      - 8083:8083
    volumes:
      - /volume1/docker/owntracks2/config:/config
      - /volume1/docker/owntracks2/store:/store
    environment:
      #- PUID=1000
      #- PGID=1000
      - TZ=America/New_York
      - OTR_USER="user"
      - OTR_PASS="pass"
      - OTR_HOST=10.1.6.17
      #Port number for MQTT (was originally 8883)
      - OTR_PORT=1883
    restart: unless-stopped

  mosquitto:
    image: eclipse-mosquitto
    networks:
      docker_ip_network:
        ipv4_address: 10.1.6.17
    ports:
      - 1883:1883
      - 8883:8883
    environment:
      #- PUID=1000
      #- PGID=1000
      - TZ=America/New_York
    volumes:
      - /volume1/docker/owntracks2/mosquitto/data:/mosquitto/data
      - /volume1/docker/owntracks2/mosquitto/logs:/mosquitto/logs
      - /volume1/docker/owntracks2/mosquitto/conf:/mosquitto/config
    restart: unless-stopped

shadowjig avatar Dec 01 '22 00:12 shadowjig

I still see

1669855031: Client otrec disconnected, not authorised.

so there’s either a mismatch in user/password or there’s an ACL forbidding the client from accessing the MQTT broker.

Also note your OTR_TOPICS setting is most likely not what you want. I would begin with owntracks/# and go from there.

I don’t see where your mosquitto.conf and passwd files are. Are you certain they’re in the the correct volume you are mounting for Mosquitto?

Please also show the Recorder’s console as you launch it.

jpmens avatar Dec 10 '22 16:12 jpmens

I confirmed both the mosquitto.conf and passwd files are in /mosquitto/config

I set OTR_TOPICS back to owntracks/#

I'm using portainer to manage docker containers. Here's the output of the log section of portainer for the recorder: No log line matching the '' filter

Where are the logs for ot-recorder located? I can't seem to find them when connected to container.

I also setup the PUID and GUID's to the admin username and group on the host machine. Also set the folder permissions to be owned by admin on the host machine. Still doesn't work. You'll notice that I have anonymous connections allowed by it's still not connecting. I don't have any ACLs. I can ping each container by IP from the other container.

shadowjig avatar Dec 11 '22 18:12 shadowjig

The console output is printed to stdout/stderr.

jpmens avatar Dec 11 '22 18:12 jpmens