zoneminder-base icon indicating copy to clipboard operation
zoneminder-base copied to clipboard

Zoneminder container starts with state STOPPED

Open tokenize47 opened this issue 2 years ago • 20 comments

There's this errors in the logs:

[zoneminder-service] 2023-02-22 14:15:02.989250625 INFO Starting ZoneMinder...
[zoneminder] 2023-02-22 14:15:11.184118221 local1.err: Feb 22 14:15:11 zmdc[922]: FAT [Can't connect to zmdc.pl server process at /zoneminder/run/zmdc.sock: No such file or directory]
[zoneminder] 2023-02-22 14:15:11.186904572 local1.err: Feb 22 14:15:11 zmpkg[916]: ERR [Unable to run "/usr/bin/zmdc.pl startup", output is "Starting server", status is 255]
[zoneminder] 2023-02-22 14:15:11.240608636 local1.err: Feb 22 14:15:11 zmpkg[916]: ERR [Unable to run "/usr/bin/zmdc.pl start zmfilter.pl --filter_id=1 --daemon", output is "Unable to connect to server using socket at /zoneminder/run/zmdc.sock", status is 255]
[zoneminder] 2023-02-22 14:15:11.292992424 local1.err: Feb 22 14:15:11 zmpkg[916]: ERR [Unable to run "/usr/bin/zmdc.pl start zmfilter.pl --filter_id=2 --daemon", output is "Unable to connect to server using socket at /zoneminder/run/zmdc.sock", status is 255]
[zoneminder] 2023-02-22 14:15:11.345458692 local1.err: Feb 22 14:15:11 zmpkg[916]: ERR [Unable to run "/usr/bin/zmdc.pl start zmwatch.pl", output is "Unable to connect to server using socket at /zoneminder/run/zmdc.sock", status is 255]
[zoneminder] 2023-02-22 14:15:11.397436936 local1.err: Feb 22 14:15:11 zmpkg[916]: ERR [Unable to run "/usr/bin/zmdc.pl start zmupdate.pl -c", output is "Unable to connect to server using socket at /zoneminder/run/zmdc.sock", status is 255]
[zoneminder] 2023-02-22 14:15:11.449087879 local1.err: Feb 22 14:15:11 zmpkg[916]: ERR [Unable to run "/usr/bin/zmdc.pl start zmtelemetry.pl", output is "Unable to connect to server using socket at /zoneminder/run/zmdc.sock", status is 255]
[zoneminder] 2023-02-22 14:15:11.501916608 local1.err: Feb 22 14:15:11 zmpkg[916]: ERR [Unable to run "/usr/bin/zmdc.pl start zmstats.pl", output is "Unable to connect to server using socket at /zoneminder/run/zmdc.sock", status is 255]
[zoneminder-service] 2023-02-22 14:15:11.506256361 INFO ZoneMinder is up! Proceeding to monitoring.

If I exec into the container and run zmpkg.pl start manually, it will then start.

tokenize47 avatar Feb 22 '23 14:02 tokenize47

I am noticing this too... Is the zmpkg.pl start missing from the /init when the container starts?

clueo8 avatar Feb 24 '23 12:02 clueo8

I am noticing this too... Is the zmpkg.pl start missing from the /init when the container starts?

No, it's not missing. The error messages comes precisely from executing "zmpkg.pl start" in the s6 init

I've done alot of debug and I really can't understand why this happens.

tokenize47 avatar Feb 24 '23 12:02 tokenize47

Can you provide some more info such as os, docker compose files, env file, etc?

alexyao2015 avatar Feb 24 '23 19:02 alexyao2015

The only changes I made from the initial git clone was just the timezone and exposed port:

.env: +TZ=America/New_York

docker-compose.yml: +9086:80

Also, my base server is x86_64 Linux 6.1.13-1-lts... I use arch btw :)

Docker version 23.0.1, build a5ee5b1dfc Docker Compose version 2.16.0

clueo8 avatar Feb 24 '23 20:02 clueo8

Honestly none of that info gives me enough information to reproduce what exactly you did and if that's unable to be done, there's no way of determining the cause other than pure speculation. As you can see, the image boots up just fine in CI and I believe many others use it without any issues. I would have to believe you must be doing something special or out of the ordinary to cause this issue.

alexyao2015 avatar Feb 25 '23 21:02 alexyao2015

I think this is an issue related with Arch, as I'm running it as well. I tried running it on Debian and it worked. I really don't understand what's causing this.

tokenize47 avatar Feb 25 '23 23:02 tokenize47

+1

also on arch, and the problems seems to come from one of the updates of the last month(s). Zoneminder runned perfectly for months, then i recently installed long waited updates ( my internet was realy bad up until two days ago, so i didn't realy update often) and then it stoped working.

at least i have a work around, thanks to @tokenize47 because of the post on the zoneminder forum i found prior to this issue here.

kubax avatar Mar 04 '23 10:03 kubax

The problem seems to only happen with Docker 23.0.1.

Docker 20.10.23 seem to work fine. So i think it will be something that has changed from docker 20.10.23 to docker 23.0.1.

Downgrading to docker 20.10.23 and rebooting (somehow restarting the service is not enough) and zoneminder is running normaly.

kubax avatar Mar 07 '23 11:03 kubax

That's a nice clue! Then the problem is not with Arch itself but the fact that Arch uses always the latest software versions.

I guess this is an incompatibility with the s6 version with the new docker version. I'll do some research on this an come up with a PR if I find a solution.

tokenize47 avatar Mar 07 '23 11:03 tokenize47

Any updates on this?

clueo8 avatar Apr 15 '23 13:04 clueo8

Problem persists with Docker 24.0.2

i have no clue how this problem could be solved.

kubax avatar Jul 16 '23 18:07 kubax

This could be the same problem as: https://github.com/zoneminder-containers/zoneminder-base/issues/50

Try upgrading from "docker-compose" to "docker compose".

Simpler1 avatar Jul 16 '23 22:07 Simpler1

Thanks for the hint.

But I don't think it is related (at least in my case).

I deploy with portainer, to a 3 cluster docker swarm (although this stack is locked to my NAS because of the hard drives)

kubax avatar Jul 16 '23 23:07 kubax

Hi all, I think the issue is caused by insufficent linux capabilities - changing the user running 'zmpkg.pl start' from www-data to root allows the socket to open (in /etc/services.d/zoneminder/run).

kondas avatar Sep 28 '23 18:09 kondas

can confirm.

as a quick test i updated to docker 24.0.5 and replaced the file inside the container with one where www-run is replaced with root.

Zoneminder starts without problems now. so at least we now know why it allways starts stopped.

thx for the update @kondas

kubax avatar Sep 28 '23 18:09 kubax

This issue appeared for me a few days ago, when I upgraded from docker 26.x to docker 27.0.3. I haven't tried the change from www-data to root yet as I don't want to it to run as root... will try to investigate what the root issue is.

gurka avatar Jul 14 '24 15:07 gurka

Had fine working setup in docker on ubuntu 24.04. But after moving to manjaro (arch based distro) i have slammed into the exactly same issue. Docker version 27.2.1 Changing www-data to root in /etc/services.d/zoneminder/run helped

kreksoverzum avatar Nov 28 '24 22:11 kreksoverzum

https://github.com/ZoneMinder/zoneminder/blob/52c41d166b7537e81c3ce14b48e0add5aa09991c/scripts/zmdc.pl.in#L273

On Debian 12, without changing www-data to root as described in previous comments, POSIX::sysconf(&POSIX::_SC_OPEN_MAX) returns 1048576 On Debian 13, without changing www-data to root, POSIX::sysconf(&POSIX::_SC_OPEN_MAX) returns 1073741816 On Debian 13, with changing www-data to root, POSIX::sysconf(&POSIX::_SC_OPEN_MAX) returns 1024

I think that the problem is that it takes too long time for that loop to execute when the value is 1073741816, and that causes it to timeout and fail here: https://github.com/ZoneMinder/zoneminder/blob/52c41d166b7537e81c3ce14b48e0add5aa09991c/scripts/zmdc.pl.in#L202

What I don't understand is, where are those values being set? Why does www-data have such a large value while root only has 1024?

This could be related: https://github.com/moby/moby/issues/44547

gurka avatar Nov 30 '24 16:11 gurka

Interesting, great diagnosis. I am tempted to just use 1024 instead of SC_OPEN_MAX. We know we only have a few fds open. We are mainly concerned with STDIN/OUT/ERR.

connortechnology avatar Nov 30 '24 17:11 connortechnology

That might be enough to fix it, yeah. There are a few more places where it iterates over all possible FDs: https://github.com/search?q=repo%3AZoneMinder%2Fzoneminder+_SC_OPEN_MAX&type=code

I still would like to figure out why it returns 1024 when changing www-data to root in /etc/services.d/zoneminder/run. If I run sysctl fs.nr_open inside the container it returns 1073741816 regardless of if I run it as root or www-data...

gurka avatar Nov 30 '24 17:11 gurka