open-balena-admin
open-balena-admin copied to clipboard
dcaputo-harmoni
Dashboard is non functional
I've tried setting-up open-balena-admin on my local machine using both docker-compose and kubernetes.
But I encounter the same error: "Non-functional dashboard".
It looks like some communication issue between open-balena-postgrest and open-balena-db from browser console error messages.
Following are the screenshots of the chrome devtools window and dashboard for your reference:
Logs from open-balena-postgrest:
07/Jun/2022:15:24:37 +0000: An error ocurred when trying to query database settings for the config parameters
07/Jun/2022:15:24:37 +0000: ConnectionError (Just "could not translate host name \"openbalena-db.openbalena\" to address: Name does not resolve\n")
07/Jun/2022:15:24:37 +0000: Attempting to connect to the database...
07/Jun/2022:15:24:37 +0000: {"details":"could not translate host name \"openbalena-db.openbalena\" to address: Name does not resolve\n","code":"","message":"Database connection error. Retrying the connection."}
07/Jun/2022:15:24:37 +0000: Attempting to reconnect to the database in 0 seconds...
> [email protected] start
> node postgrest-proxy.js
[HPM] Proxy created: / -> http://127.0.0.1:3000
07/Jun/2022:15:24:38 +0000: {"details":"could not translate host name \"openbalena-db.openbalena\" to address: Name does not resolve\n","code":"","message":"Database connection error. Retrying the connection."}
07/Jun/2022:15:24:38 +0000: Attempting to reconnect to the database in 1 seconds...
07/Jun/2022:15:24:40 +0000: {"details":"could not translate host name \"openbalena-db.openbalena\" to address: Name does not resolve\n","code":"","message":"Database connection error. Retrying the connection."}
07/Jun/2022:15:24:40 +0000: Attempting to reconnect to the database in 2 seconds...
07/Jun/2022:15:24:44 +0000: {"details":"could not translate host name \"openbalena-db.openbalena\" to address: Name does not resolve\n","code":"","message":"Database connection error. Retrying the connection."}
07/Jun/2022:15:24:44 +0000: Attempting to reconnect to the database in 4 seconds...
[HPM] Error occurred while proxying request postgrest.openbalena.abc/user?limit=10&offset=0&order=id.asc to http://127.0.0.1:3000/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
07/Jun/2022:15:24:52 +0000: {"details":"could not translate host name \"openbalena-db.openbalena\" to address: Name does not resolve\n","code":"","message":"Database connection error. Retrying the connection."}
07/Jun/2022:15:24:52 +0000: Attempting to reconnect to the database in 8 seconds...
[HPM] Error occurred while proxying request postgrest.openbalena.abc/user?limit=10&offset=0&order=id.asc to http://127.0.0.1:3000/ [ECONNREFUSED] (https://nodejs.org/api/errors.html#errors_common_system_errors)
07/Jun/2022:15:25:08 +0000: {"details":"could not translate host name \"openbalena-db.openbalena\" to address: Name does not resolve\n","code":"","message":"Database connection error. Retrying the connection."}
07/Jun/2022:15:25:08 +0000: Attempting to reconnect to the database in 16 seconds...
Logs from open-balena-db:
2022-06-07 14:49:50.879 UTC [6442] LOG: invalid length of startup packet
...
2022-06-07 15:16:56.159 UTC [6776] LOG: invalid length of startup packet
@dcaputo-harmoni Can you please help me resolve this issue?
This looks like a docker networking issue - can you ssh into the open-balena-postgrest container and run ping openbalena-db? I suspect it will not resolve, in which case you need to confirm that the open-balena-admin containers are running within the same docker network as open-balena.
You might want to take a look at the services.yaml file, which you'll see puts all of the open-balena-admin containers on the openbalena_default network, which is the default network used by open-balena. If you have changed this, or if you are running your open-balena on a different docker server than open-balena-admin, you'll need to somehow ensure that the open-balena-admin containers can communicate with the open-balena containers.
Thanks for the reply @dcaputo-harmoni. Sorry for the confusion. Your point might be helpful in case I'm running them using docker. But I'm running both open-balena and open-balena-admin using kubernetes, where both of them are running on the same default namespace.
FYI: I'm seeing the following when I do the ping in kubernetes:
root@openbalena-admin-postgrest-557b7fbcfb-c7wg9:/usr/src/app# ping openbalena-db
PING openbalena-db.default.svc.cluster.local (10.107.164.210) 56(84) bytes of data.
^C
--- openbalena-db.default.svc.cluster.local ping statistics ---
80 packets transmitted, 0 received, 100% packet loss, time 80879ms
Following is the output when I do minikube kubectl get services:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4d22h
openbalena-admin-haproxy LoadBalancer 10.105.171.63 10.105.171.63 80:31659/TCP,443:30041/TCP,10000:32538/TCP,10001:30731/TCP,10002:30684/TCP,10003:32040/TCP,10004:31675/TCP,10005:30778/TCP,10006:32599/TCP,10007:31427/TCP,10008:31011/TCP,10009:31224/TCP 4d21h
openbalena-admin-postgrest ClusterIP 10.104.204.79 <none> 80/TCP 4d21h
openbalena-admin-remote ClusterIP 10.110.96.146 <none> 80/TCP,10000/TCP,10001/TCP,10002/TCP,10003/TCP,10004/TCP,10005/TCP,10006/TCP,10007/TCP,10008/TCP,10009/TCP 4d21h
openbalena-admin-ui ClusterIP 10.108.227.99 <none> 80/TCP 4d21h
openbalena-api ClusterIP 10.96.16.121 <none> 80/TCP 4d22h
openbalena-db ClusterIP 10.107.164.210 <none> 5432/TCP 4d22h
openbalena-haproxy LoadBalancer 10.107.201.237 10.107.201.237 80:30564/TCP,443:31031/TCP 4d22h
openbalena-internal-tunnel ClusterIP 10.103.153.81 <none> 3128/TCP 4d22h
openbalena-redis ClusterIP 10.104.156.196 <none> 6379/TCP 4d22h
openbalena-registry ClusterIP 10.109.61.80 <none> 80/TCP 4d22h
openbalena-s3 ClusterIP 10.98.253.0 <none> 80/TCP 4d22h
openbalena-vpn ClusterIP 10.110.117.105 <none> 443/TCP,3128/TCP 4d22h
As you can see above openbalena-db doesn't have an external ip.
Am I missing any other configuration to get it to work?
This looks like a namespace issue. Notice the error message is looking for openbalena-db.openbalena but you are using openbalena-db.default. You will either need to change the service config files to point to the default namespace (change references to openbalena-db.default) or put openbalena in the openbalena namespace.
By the way - nice work getting all this up and running in kubernetes.