Couch soft crashes and restart fails to bind to port

[digimbyte]

Description

CouchDB service is running but has failed to bind to port even after restarting the service

Steps to Reproduce

install on Debian Buster, setup with standard dedicated database

Expected Behaviour

Server to be active, bound to port and reporting well when I run status

Your Environment

NGINX setup for front websites the port is open, have used CouchDB web panel before after possible crash or restart - software will not bind to port

this is the second time it has happened, last time it was fixed by resetting the admin password.

• CouchDB version used: 3.2.0
• Browser name and version: ANY
• Operating system and version: ANY

Additional Context

I can't seem to get logging to log anything

output from console:

admin@server:/opt$ sudo netstat -tulpn | grep LISTEN
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      5972/nginx: worker
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      563/sshd
tcp6       0      0 :::2080                 :::*                    LISTEN      6854/node
tcp6       0      0 :::4080                 :::*                    LISTEN      6846/node
tcp6       0      0 :::22                   :::*                    LISTEN      563/sshd

admin@server:~$ /etc/init.d/couchdb status
● couchdb.service - Apache CouchDB
   Loaded: loaded (/lib/systemd/system/couchdb.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2022-03-13 04:51:12 UTC; 15ms ago
 Main PID: 32309 (couchdb)
    Tasks: 3 (limit: 2330)
   Memory: 612.0K
   CGroup: /system.slice/couchdb.service
           ├─32309 /bin/sh /opt/couchdb/bin/couchdb
           ├─32317 /bin/sh /opt/couchdb/bin/couchdb
           └─32319 /bin/sh /opt/couchdb/bin/couchdb

admin@server:~$ curl http://127.0.0.1:5984/
curl: (7) Failed to connect to 127.0.0.1 port 5984: Connection refused

[nickva]

@digimbyte thanks for your report. Is it easy to reproduce. Something like: make sure it's running, then kill its main process and when it restarts this happens?

The resetting the admin password. part stood out a bit. Did that involve a manual restart?

Check and see what is running or bound to that port when the error happens. That is to see if it's a permissions issue (like a firewall rule) or because something else is bound to it. Could it be that the old instance is still running and bound to it or another CouchDB instance (in a container environment)?

[trames57]

I am experiencing something like this when I upgraded to 3.2.2-2 (standalone). I get a "Protocol 'inet_tcp' : register/listen error: econnrefused" "epmd: failed to bind on ipaddr ::1" error in journalctl on the couchdb service. On this server, I have IPV6 disabled (I was having issues with the javascript nano library connections). I downgraded to 3.2.2-1 and everything works fine. The issue is on a Linode server. I did do an lsof -i -P -n, and there was nothing bound to port 5984. After I downgraded I did the lsof again, and it was bound as it should be.

I then reenabled IPV6 and the 3.2.2-2 is working, but now the javascript nano connection is having issues again. I need "epmd" to bind to IPV4, but not sure how to accomplish that.

Update: I did follow the epmd erlang trail. The couchdb.service file is setting the environment variable ERL_EPMD_ADDRESS to 127.0.0.1, but epmd does not appear to be using it.

Update 2: This may be an issue with changes to the couchdb.service file on Arch Linux. They changed the Environment parameter to the EnvironmentFile parameter which has the ERL_EPMD_ADDRESS in /etc/default/couchdb. When I removed the env. variable from /etc/default/couchdb, the server started without errors. This env. var was NOT in the previous version's Environment parameter.

Arch Linux - kernel 5.18.2

[big-r81]

Hi, so it's an Arch Linux behavior?

[trames57]

I put a bug post into arch linux but no one has responded to it yet. I would say the epmd issue not using the environment variable as described is not an arch issue. Given that it is 127.0.0.1, it should not be trying with ::1

[digimbyte]

I had this issue on Debian Buster as well - in a production environment. was catastrophic

On Tue, Jul 5, 2022 at 9:42 AM trames57 @.***> wrote:

I put a bug post into arch linux but no one has responded to it yet. I would say the epmd issue not using the environment variable as described is not an arch issue. Given that it is 127.0.0.1, it should not be trying with ::1

— Reply to this email directly, view it on GitHub https://github.com/apache/couchdb/issues/3958#issuecomment-1174461771, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSWNFEM7CNLAWH5P7KFTLLVSNZH3ANCNFSM5QS3OWRA . You are receiving this because you authored the thread.Message ID: @.***>