mosquitto icon indicating copy to clipboard operation
mosquitto copied to clipboard

Mosquitto not working anymore after Synology update

Open AlbertHoevenaars opened this issue 1 year ago • 14 comments

After updating my Synology NAS DS1019+ last week, Mosquitto will not start anymore. Before the update Mosquitto was working (within Docker) without any problem. The message I get is "driver failed programming external connectivity on endpoint mosquitto (178c4e06e5edb9aad649bc2789e5ce7c160fd2b85daaae5b599cebd22a0950a6): Error starting userland proxy: listen tcp4 0.0.0.0:1883: bind: address already in use" my mosquitto.conf file holds:

allow_anonymous false listener 1883 192.168.2.160

persistence true persistence_location /mosquitto/data/ log_dest file /mosquitto/log/mosquitto.log

password_file /mosquitto/config/password_file

No log file is created. By the way I have never seen something in the log directory.

How to solve this problem and get Mosquitto working again?

AlbertHoevenaars avatar Aug 07 '23 11:08 AlbertHoevenaars

Untitled When I start Mosquitto in Docker this is the message I got

AlbertHoevenaars avatar Aug 07 '23 12:08 AlbertHoevenaars

From the error message Error starting userland proxy: listen tcp4 0.0.0.0:1883: bind: address already in use it sounds like docker is not even be able to start the container. This is an error message from docker and not from the mosquitto running inside the container. The userland proxy is part of docker used to forward incoming connections into the container. On a linux system user land proxy are most times not really required, because network forwarding for the docker containers is achieved using the iptables kernel module.

But anyway this message means the is still a different process listening to the port 1883, which docker would like to bind for the Mosquitto broker. Not sure, which tools are installed in the NAS and if you are able to log into it. Easiest way to find out the process would be using a netstat -nap | grep 1883 executed as root. Alternative would be to use a lsof command.

My best guess would be something went wrong with the docker update/config. I have seen problems with containers not really being stopped (reboot should fix that). Or problems with different docker installation existing at the same time (as there are most times different ways to install docker).

NorbertHeusser avatar Aug 28 '23 08:08 NorbertHeusser

Norbert, thanks for the message. I executed the command and the result is attached. Screenshot 2023-08-28 112140. Because I am not experienced in these things, maybe you can tell me what's wrong and how to correct it.

AlbertHoevenaars avatar Aug 28 '23 09:08 AlbertHoevenaars

Any update on this? I have the same issue on my Synology DS918+ running DSM 7.2.1 Update 3. @AlbertHoevenaars did you solve it?

Sir-Bacon avatar Apr 23 '24 07:04 Sir-Bacon

@Sir-Bacon, I'm not familiar with synology in general but if you run sudo netstat -lntp | grep 1883 you should see what is listening on port 1883 - preventing the mosquitto container from starting.

If it happens to be "docker-proxy", there should be another process in docker occupying the port;

If you are able to run docker commands from cli, then you can try docker ps -a or sudo docker ps -a to get a list of containers and what ports they listen on.

Did mosquitto update in this process as well, by any chanse?

I have a hunch that it's going to be a mosqutto (stopped or started), that is listening on it, but with a bad configuration.

  • just a guess.

@AlbertHoevenaars problem is probably due to the fact listener 1883 192.168.2.160 specifies an ip - most likely not used by the container, rather the host.

just listener 1883 should do fine.

I would also reorder the config to look more like:

persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log

listener 1883
allow_anonymous false
password_file /mosquitto/config/password_file

given that some options are listener specific thus can start the "default listener", which is not what you want to do here.

Daedaluz avatar Apr 23 '24 12:04 Daedaluz

@Daedaluz : Thanks for the help. I run the command, check the screenshot below. image

There is no Docker active on port 1883. It is listed as nginx. I am not aware of any nginx running on my Docker. I do not use reverse proxy, don't have any other docker running, so have no clue what this is.

Sir-Bacon avatar Apr 23 '24 19:04 Sir-Bacon

What on earth is a nginx doing on port 1883?! Some websocket mqtt attempt perhaps?

Daedaluz avatar Apr 23 '24 19:04 Daedaluz

Perhaps this was a left-over from an earlier attempt to install the Eclipse docker. In the new Synology Container Manager the first selection you can do is to setup a web portal: image

I may have done this on an earlier attempt. I have now deleted that in the Web Station web portal, no more ports in use there. Checked with Putty netstat, nothing on 1883.

Did a new mqtt container without webstation but with normal port settings: image

When I start the container, it still immediately stops. This is the log: image

Sir-Bacon avatar Apr 23 '24 19:04 Sir-Bacon

now, how does your mosquitto config look like?

Daedaluz avatar Apr 23 '24 19:04 Daedaluz

persistence true persistence_location /mosquitto/data log_dest file /mosquitto/log/mosquitto.log #password_file /mosquitto/config/pwfile allow_anonymous true listener 1883 192.168.2.31

Sir-Bacon avatar Apr 23 '24 20:04 Sir-Bacon

Can you try the config file i provided previously?

192.168.2.31 is not available inside the container

Daedaluz avatar Apr 23 '24 20:04 Daedaluz

Yep, that worked! Thanks so much! (Only little change I have is that allow_anonymous true is used)

Sir-Bacon avatar Apr 23 '24 20:04 Sir-Bacon

just keep that one under the listener part.

Daedaluz avatar Apr 23 '24 20:04 Daedaluz

I did. Thanks again!

Sir-Bacon avatar Apr 23 '24 20:04 Sir-Bacon