couchdb-pkg icon indicating copy to clipboard operation
couchdb-pkg copied to clipboard

weatherreport not working with debian package

Open sblaisot opened this issue 2 years ago • 6 comments

Trying to use weatherreport in couchdb installation form deb package leads to errors

Expected Behavior

$ /opt/couchdb/bin/weatherreport --list
Available diagnostic checks:

  custodian            Shard safety/liveness checks
  disk                 Data directory permissions and atime
  internal_replication Check the number of pending internal replication jobs
  ioq                  Check the total number of active IOQ requests
  mem3_sync            Check there is a registered mem3_sync process
  membership           Cluster membership validity
  memory_use           Measure memory usage
  message_queues       Check for processes with large mailboxes
  node_stats           Check useful erlang statistics for diagnostics
  nodes_connected      Cluster node liveness
  process_calls        Check for large numbers of processes with the same current/initial call
  process_memory       Check for processes with high memory usage
  safe_to_rebuild      Check whether the node can safely be taken out of service
  search               Check the local search node is responsive
  tcp_queues           Measure the length of tcp queues in the kernel

Current Behavior

$ /opt/couchdb/bin/weatherreport --list
/usr/bin/env: 'escript': No such file or directory

$ /opt/couchdb/bin/weatherreport --etc /opt/couchdb/etc --list
/usr/bin/env: 'escript': No such file or directory

$ PATH=/opt/couchdb/bin:/opt/couchdb/erts-12.3.2.10/bin:$PATH
$ /opt/couchdb/bin/weatherreport --etc /opt/couchdb/etc --list
{"init terminating in do_boot",{'cannot get bootfile','no_dot_erlang.boot'}}
init terminating in do_boot ({cannot get bootfile,no_dot_erlang.boot})

Crash dump is being written to: erl_crash.dump...done

Possible Solution

Document instructions on how to use weatherreport

Steps to Reproduce (for bugs)

  1. Install couchdb from deb package on a fresh system
  2. Try to use weatherreport with above commands

Context

Your Environment

Couchdb 3.3.2 deb package on Debian 11 Buulseye

# cat /etc/debian_version 
11.8

# dpkg -l couchdb
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version        Architecture Description
+++-==============-==============-============-==================================
ii  couchdb        3.3.2~bullseye amd64        RESTful document oriented database


# curl -fsSL http://127.0.0.1:5984/ | jq .
{
  "couchdb": "Welcome",
  "version": "3.3.2",
  "git_sha": "11a234070",
  "uuid": "f3fd9691d6ac51fae969a4701fab4cc0",
  "features": [
    "access-ready",
    "partitioned",
    "pluggable-storage-engines",
    "reshard",
    "scheduler"
  ],
  "vendor": {
    "name": "The Apache Software Foundation"
  }
}

sblaisot avatar Nov 22 '23 10:11 sblaisot

Thanks for your report @sblaisot. Yeah this looks like a bug where we assume there a system-wide Erlang distribution (we expect escript binary to be in PATH) but our Apache CouchDB release is installed in /opt/couch currently.

nickva avatar Nov 25 '23 04:11 nickva

@nickva having escript in path is not sufficient by itself

$ PATH=/opt/couchdb/bin:/opt/couchdb/erts-12.3.2.10/bin:$PATH
$ /opt/couchdb/bin/weatherreport --etc /opt/couchdb/etc --list
{"init terminating in do_boot",{'cannot get bootfile','no_dot_erlang.boot'}}
init terminating in do_boot ({cannot get bootfile,no_dot_erlang.boot})

Crash dump is being written to: erl_crash.dump...done

sblaisot avatar Dec 12 '23 08:12 sblaisot

Thanks @sblaisot. I imagine we may have to do something similar to what our remsh script does: https://github.com/apache/couchdb/blob/main/rel/overlay/bin/remsh

nickva avatar Dec 13 '23 03:12 nickva

@nickva do you have any hint on how to run weather-report from deb install?

currently I'm not able to run it.

sblaisot avatar Jan 12 '24 16:01 sblaisot

@sblaisot

I can get it to run but then it can't connect to the node

export PATH="${PATH}:/opt/couchdb/bin:/opt/couchdb/erts-12.3.2.15/bin/"
cd /opt/couchdb/releases/3.3.3
# Point no_dot_erlang.boot to start_clean.boot
sudo ln -s start_clean.boot no_dot_erlang.boot
/opt/couchdb/bin/weatherreport --etc /opt/couchdb/etc/

baylisscg avatar Mar 20 '24 02:03 baylisscg

Is there a solution or fix for this available? The workaround from @baylisscg works for me to and i too can't connect to the local node.

root@db01:/opt/couchdb/releases/3.3.3# /opt/couchdb/bin/weatherreport --etc /opt/couchdb/etc/ -d debug
[nonode@nohost] [debug] Reading config from files: ["/opt/couchdb/etc/default.ini",
                            "/opt/couchdb/etc/local.ini"]
[nonode@nohost] [debug] Local node config: [{{"chttpd","bind_address"},"127.0.0.1"},
                    {{"chttpd","port"},"5984"},
                    {{"chttpd_auth","hash_algorithms"},"sha256, sha"},
                    {{"couch_httpd_auth","authentication_db"},"_users"},
                    {{"couchdb","database_dir"},"/HDD02/couchNew/data"},
                    {{"couchdb","view_index_dir"},"/HDD02/couchNew/data"},
                    {{"couchdb_engines","couch"},"couch_bt_engine"},
                    {{"feature_flags","partitioned||*"},"true"},
                    {{"httpd","bind_address"},"127.0.0.1"},
                    {{"httpd","port"},"5986"},
                    {{"indexers","couch_mrview"},"true"},
                    {{"prometheus","additional_port"},"false"},
                    {{"prometheus","bind_address"},"127.0.0.1"},
                    {{"prometheus","port"},"17986"},
                    {{"smoosh","state_dir"},"/HDD02/couchNew/data"},
                    {{"vendor","name"},"The Apache Software Foundation"}]

[nonode@nohost] [debug] Not connected to the local cluster node, trying to connect. alive:false connect_failed:undefined
[nonode@nohost] [debug] Starting distributed Erlang.
['[email protected]'] [warning] Could not connect to the local cluster node '[email protected]', some checks will not run.
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [debug] Local RPC: rpc:multicall([['[email protected]'],
                          erlang,apply,
                          [#Fun<weatherreport_runner.1.90675952>,[]],
                          300000]) [300000]
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_tcp_queues: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_search: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_safe_to_rebuild: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_process_memory: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_process_calls: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_nodes_connected: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_node_stats: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_message_queues: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_memory_use: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_membership: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_mem3_sync: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_ioq: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_internal_replication: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_disk: nodedown
['[email protected]'] [crit] Bad rpc call executing check weatherreport_check_custodian: nodedown

jonasplaum avatar Apr 25 '24 12:04 jonasplaum