tak-server icon indicating copy to clipboard operation
tak-server copied to clipboard

Cannot reach DB when running setup script (TAK 5.0 and 5.1)

Open SudomenLeah opened this issue 1 year ago • 8 comments

No joy with DB at (redacted), will retry in 10s. If this loops more than 6 times go and get some fresh air...
java.lang.reflect.InvocationTargetException
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
      at com.bbn.marti.takcl.cli.CommandCommon.invokeMethodFromModuleWithParams(CommandCommon.java:85)
      at com.bbn.marti.takcl.cli.advanced.AdvancedParamParser.execute(AdvancedParamParser.java:285)
      at com.bbn.marti.takcl.cli.advanced.AdvancedCliMainHelper.main(AdvancedCliMainHelper.java:131)
      at com.bbn.marti.UserManager.main(UserManager.java:61)
Caused by: java.lang.reflect.InvocationTargetException
      at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
      at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.base/java.lang.reflect.Method.invoke(Method.java:568)
      at com.bbn.marti.takcl.cli.CommandCommon.invokeMethodFromModuleWithParams(CommandCommon.java:52)
      ... 3 more
Caused by: class org.apache.ignite.IgniteClientDisconnectedException: Failed to execute task, client node disconnected.
      at org.apache.ignite.internal.util.IgniteUtils$15.apply(IgniteUtils.java:1046)
      at org.apache.ignite.internal.util.IgniteUtils$15.apply(IgniteUtils.java:1042)
      at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1145)
      at org.apache.ignite.internal.processors.service.GridServiceProxy.invokeMethod(GridServiceProxy.java:257)
      at org.apache.ignite.internal.processors.service.GridServiceProxy$ProxyInvocationHandler.invoke(GridServiceProxy.java:511)
      at jdk.proxy2/jdk.proxy2.$Proxy40.addOrUpdateUser(Unknown Source)
      at com.bbn.marti.takcl.AppModules.OnlineFileAuthModule.innerUserCertMod(OnlineFileAuthModule.java:189)
      at com.bbn.marti.takcl.AppModules.OnlineFileAuthModule.usermod(OnlineFileAuthModule.java:91)
      ... 8 more
Caused by: class org.apache.ignite.internal.IgniteClientDisconnectedCheckedException: Failed to execute task, client node disconnected.
      at org.apache.ignite.internal.processors.task.GridTaskProcessor.disconnectedError(GridTaskProcessor.java:230)
      at org.apache.ignite.internal.processors.task.GridTaskProcessor.onDisconnected(GridTaskProcessor.java:218)
      at org.apache.ignite.internal.IgniteKernal.onDisconnected(IgniteKernal.java:3166)
      at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$3.onDiscovery0(GridDiscoveryManager.java:831)
      at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$3.access$7300(GridDiscoveryManager.java:547)
      at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$3$NotificationTask.run(GridDiscoveryManager.java:980)
      at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.java:2822)
      at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java:2860)
      at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
      at java.base/java.lang.Thread.run(Thread.java:833)
java.lang.reflect.InvocationTargetException
No joy with DB at (redacted), will retry in 10s. If this loops more than 6 times go and get some fresh air...

Environment

Please provide as much information as possible about your environment:

  • Hardware Make/Model: linode VPS
  • Operating System and Version (e.g.cat /etc/os-release`) :Ubuntu 20.04 LTS
  • Docker Version (docker -v): Docker version 26.0.2,
  • Docker Compose Version (docker compose version) or (docker-compose version): Docker Compose version v2.26.1
  • CPU Details (make/model and/or speed/core count): 2 cores
  • RAM (e.g. free -h): 4GB RAM
  • Disk Availability (e.g. df -h):
  • TAK Server Version From TAK.gov: 5.0 & also tried 5.1

Description

When running the setup script postgres cannot be reached. getting this java error. (see above)

Steps to Reproduce

run the setup script

SudomenLeah avatar Apr 20 '24 19:04 SudomenLeah

I have the same issue. tak-server-db is stuck in restarting state

c9dd8cbf0f8 tak-server-db "/bin/bash -c /opt/t…" About an hour ago Restarting (126) 10 seconds ago tak-server-db-1

NoOneAtHome avatar Apr 24 '24 21:04 NoOneAtHome

I'm also having the same issue on Ubuntu 22.04 LTS Any update?

Jblue117 avatar May 09 '24 16:05 Jblue117

I just did a successful install of 5.1 on a Ubuntu 22.04 LTS VPS last night. It looped through the "no joy" 3 times then continued successfully. Server specs: 16 GB ram, 4 cpus, SSD

hypeitinc avatar May 14 '24 17:05 hypeitinc

I am also having this issue. I'm on a fresh install of Debian 12. It looped through the check 6 times and failed. I have the same java.lang.reflect.InvocationTargetException error as SudomenLeah in the code block above. This is a problem for me for all TAK versions: 4.10, 5.0 and 5.1.

celsian avatar May 23 '24 19:05 celsian

I think I got it to work. After looking at the docker logs for the db I was seeing an error for permission denied for /opt/tak/db-utils/configureInDocker.sh.

So I did the following to make it work: cp -r /tak-server/tak /opt sudo chmod -x /opt/tak/db-utils/*

After this I ran the setup.sh with sudo in front to be safe and it came up with no problems.

js98947 avatar May 25 '24 02:05 js98947

I decided to spin up a VM instead and got it working that way. It's possible the VM I was trying to install the docker container on had too little memory to start as I was getting the same error in the VM despite allowing it to expand to 8gb of memory. After forcing the container to 8gb the installation and postgres setup worked.

celsian avatar May 25 '24 13:05 celsian

Increasing memory seems to be the recommended solution but why this needs 8GB just to start is hard to justify but when you need to respin the server to revoke a user it's to be expected.

FarrantAlex avatar Jul 11 '24 11:07 FarrantAlex

Same issue, I am trying with 5.1, but same error is coming. Can it be a problem if I am running this on Mac?

om-livebird avatar Jul 25 '24 09:07 om-livebird