apitable icon indicating copy to clipboard operation
apitable copied to clipboard
verified publisher icon apitable

question: Database connection fails in anycase

Open bytesnackscgn opened this issue 3 years ago • 3 comments

OS: Ubuntu 22 Firewall: UFW but it is disabled.

Mysql Container starts and returns that it is ready for conncetions. "/usr/sbin/mysqld: ready for connections. Version: '8.0.32' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL."

But i can not connect from outside the server to the container. Accesing the container via exec works and also accessing the containers database. BUT not from outside. And this causes that pre-initdb container cannot complete its process and fails with:

Caused by: liquibase.exception.DatabaseException: Connection could not be created to jdbc:mysql://myip:3306/apitable?characterEncoding=utf8&autoReconnect=true&useSSL=true with driver com.mysql.cj.jdbc.Driver. Could not create connection to database server. Attempted reconnect 3 times. Giving up. compose-pre-init-db-1 | at liquibase.database.jvm.JdbcConnection.open(JdbcConnection.java:54) compose-pre-init-db-1 | at com.datical.liquibase.ext.database.jvm.ProJdbcConnection.open(Unknown Source) compose-pre-init-db-1 | at liquibase.database.ConnectionServiceFactory.create(ConnectionServiceFactory.java:32)

Is there any config from docker which i am missing?

Have spent more than 8 hours and it is just driving me crazy... :D

bytesnackscgn avatar Feb 25 '23 18:02 bytesnackscgn

I'm not an official, but I configured the environment correctly through the make dataenv command provided by their developer guide.

ziqiangai avatar Feb 26 '23 06:02 ziqiangai

@bytesnackscgn

Please verify the network connectivity between containers:

$ docker compose exec -it room-server bash
root@35f65c3d6087:/app# curl mysql:3306
curl: (1) Received HTTP/0.9 when not allowed
root@35f65c3d6087:/app# curl redis:6379
curl: (52) Empty reply from server

networkhermit avatar Feb 27 '23 01:02 networkhermit

Hey i was just struggling with the bridge mode. Have forgot that docker links the containers in local network without exposing the ports to public and i had replaced the servicenames with my ip in ENV file.

I get the responses from youre answer @networkhermit

Now all containers start and are healthy but breaks on init app data so something is wrong with the redis container:

log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:00.022"}\n","stream":"stdout","time":"2023-02-27T20:06:00.027103834Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:00.023"}\n","stream":"stdout","time":"2023-02-27T20:06:00.027108332Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:00.026"}\n","stream":"stdout","time":"2023-02-27T20:06:00.027112059Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:00.026"}\n","stream":"stdout","time":"2023-02-27T20:06:00.027115576Z"} {"log":"{"args":"transport error","context":"RedisIoAdapter","level":"info","message":"RedisIoAdapter:clientDisconnecting","socketId":"HggCoGo7X6IMzAcaAAAc","timestamp":"2023-02-27 20:06:04.807","userId":"java_7079ecf35012/192.168.0.9"}\n","stream":"stdout","time":"2023-02-27T20:06:04.808588607Z"} {"log":"{"args":"transport error","context":"RedisIoAdapter","level":"warn","message":"RedisIoAdapter:clientDisconnect","socketId":"HggCoGo7X6IMzAcaAAAc","timestamp":"2023-02-27 20:06:04.807","userId":"java_7079ecf35012/192.168.0.9"}\n","stream":"stdout","time":"2023-02-27T20:06:04.808620888Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:05.008"}\n","stream":"stdout","time":"2023-02-27T20:06:05.008736588Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:05.008"}\n","stream":"stdout","time":"2023-02-27T20:06:05.009339792Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:05.010"}\n","stream":"stdout","time":"2023-02-27T20:06:05.011026736Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:05.013"}\n","stream":"stdout","time":"2023-02-27T20:06:05.013951518Z"} {"log":"{"context":"RedisIoAdapter","level":"debug","message":"RedisIoAdapter:clientConnect","nsp":"/","socketId":"xeieKWls0Mm3FD5tAAAd","timestamp":"2023-02-27 20:06:05.089","userId":"java_7079ecf35012/192.168.0.9"}\n","stream":"stdout","time":"2023-02-27T20:06:05.090335209Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:10.008"}\n","stream":"stdout","time":"2023-02-27T20:06:10.010043631Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:10.011"}\n","stream":"stdout","time":"2023-02-27T20:06:10.011659059Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:10.011"}\n","stream":"stdout","time":"2023-02-27T20:06:10.012341955Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:10.012"}\n","stream":"stdout","time":"2023-02-27T20:06:10.013136373Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:15.008"}\n","stream":"stdout","time":"2023-02-27T20:06:15.009447167Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:15.011"}\n","stream":"stdout","time":"2023-02-27T20:06:15.011738817Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:15.012"}\n","stream":"stdout","time":"2023-02-27T20:06:15.012608268Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:15.012"}\n","stream":"stdout","time":"2023-02-27T20:06:15.01325206Z"} {"log":"{"args":"transport error","context":"RedisIoAdapter","level":"info","message":"RedisIoAdapter:clientDisconnecting","socketId":"xeieKWls0Mm3FD5tAAAd","timestamp":"2023-02-27 20:06:15.095","userId":"java_7079ecf35012/192.168.0.9"}\n","stream":"stdout","time":"2023-02-27T20:06:15.096228687Z"} {"log":"{"args":"transport error","context":"RedisIoAdapter","level":"warn","message":"RedisIoAdapter:clientDisconnect","socketId":"xeieKWls0Mm3FD5tAAAd","timestamp":"2023-02-27 20:06:15.095","userId":"java_7079ecf35012/192.168.0.9"}\n","stream":"stdout","time":"2023-02-27T20:06:15.09626703Z"} {"log":"{"context":"RedisIoAdapter","level":"debug","message":"RedisIoAdapter:clientConnect","nsp":"/","socketId":"vExcrejEAK7ovtceAAAd","timestamp":"2023-02-27 20:06:15.673","userId":"java_7079ecf35012/192.168.0.9"}\n","stream":"stdout","time":"2023-02-27T20:06:15.673819955Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:20.007"}\n","stream":"stdout","time":"2023-02-27T20:06:20.008210507Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:20.007"}\n","stream":"stdout","time":"2023-02-27T20:06:20.008429684Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:20.008"}\n","stream":"stdout","time":"2023-02-27T20:06:20.009521887Z"} {"log":"{"context":"ClientProxy","level":"info","message":"Available IP Pools：192.168.0.8:3334","timestamp":"2023-02-27 20:06:20.009"}\n","stream":"stdout","time":"2023-02-27T20:06:20.010209572Z"}

bytesnackscgn avatar Feb 27 '23 20:02 bytesnackscgn

@bytesnackscgn Sorry. The log messages you provided are not from init-appdata. If you modified the .env please run docker compose up -d to make it effective.

networkhermit avatar Mar 03 '23 07:03 networkhermit

More information: I am trying to run minio container locally not in aws. So i have inserted: OSS_CLIENT_TYPE=minio

But with oss enabled it tries still via aws and tries to access via default values. Is there something what i do not have in mind? is aws required and i have not read it in instructions?

compose-init-appdata-1 | com.amazonaws.services.s3.model.AmazonS3Exception: The Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: 1749F38908D36438; S3 Extended Request ID: aacd977c-0831-498a-8cc0-eda03418482d; Proxy: null), S3 Extended Request ID: aacd977c-0831-498a-8cc0-eda03418482d compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879) compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418) compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387) compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157) compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814) compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781) compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755) compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715) compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697) compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561) compose-init-appdata-1 | at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541) compose-init-appdata-1 | at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456) compose-init-appdata-1 | at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403) compose-init-appdata-1 | at com.amazonaws.services.s3.AmazonS3Client.getAcl(AmazonS3Client.java:4062) compose-init-appdata-1 | at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1278) compose-init-appdata-1 | at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1268) compose-init-appdata-1 | at com.amazonaws.services.s3.AmazonS3Client.doesBucketExistV2(AmazonS3Client.java:1406) compose-init-appdata-1 | at com.apitable.appdata.shared.starter.oss.AwsOssClientRequest.isBucketExist(AwsOssClientRequest.java:42) compose-init-appdata-1 | at com.apitable.appdata.shared.starter.oss.AwsOssClientRequest.uploadStreamForObject(AwsOssClientRequest.java:66) compose-init-appdata-1 | at com.apitable.appdata.shared.starter.oss.OssClientTemplate.upload(OssClientTemplate.java:28) compose-init-appdata-1 | at com.apitable.appdata.loader.service.impl.AppDataLoaderServiceImpl.loadAsset(AppDataLoaderServiceImpl.java:75) compose-init-appdata-1 | at com.apitable.appdata.loader.service.impl.AppDataLoaderServiceImpl$$FastClassBySpringCGLIB$$65b15b21.invoke() compose-init-appdata-1 | at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) compose-init-appdata-1 | at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386) compose-init-appdata-1 | at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85) compose-init-appdata-1 | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:704) compose-init-appdata-1 | at com.apitable.appdata.loader.service.impl.AppDataLoaderServiceImpl$$EnhancerBySpringCGLIB$$477ee4e8.loadAsset() compose-init-appdata-1 | at com.apitable.appdata.command.AdmCommand$Load.call(AdmCommand.java:52) compose-init-appdata-1 | at com.apitable.appdata.command.AdmCommand$Load.call(AdmCommand.java:43) compose-init-appdata-1 | at picocli.CommandLine.executeUserObject(CommandLine.java:2041) compose-init-appdata-1 | at picocli.CommandLine.access$1500(CommandLine.java:148) compose-init-appdata-1 | at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461) compose-init-appdata-1 | at picocli.CommandLine$RunLast.handle(CommandLine.java:2453) compose-init-appdata-1 | at picocli.CommandLine$RunLast.handle(CommandLine.java:2415) compose-init-appdata-1 | at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273) compose-init-appdata-1 | at picocli.CommandLine$RunLast.execute(CommandLine.java:2417) compose-init-appdata-1 | at picocli.CommandLine.execute(CommandLine.java:2170) compose-init-appdata-1 | at com.apitable.appdata.command.AdmCommandRunner.run(AdmCommandRunner.java:34) compose-init-appdata-1 | at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:768) compose-init-appdata-1 | at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:752) compose-init-appdata-1 | at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) compose-init-appdata-1 | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) compose-init-appdata-1 | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) compose-init-appdata-1 | at com.apitable.appdata.AppDataApplication.main(AppDataApplication.java:12) compose-init-appdata-1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) compose-init-appdata-1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) compose-init-appdata-1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) compose-init-appdata-1 | at java.lang.reflect.Method.invoke(Method.java:498) compose-init-appdata-1 | at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) compose-init-appdata-1 | at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) compose-init-appdata-1 | at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) compose-init-appdata-1 | at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) compose-init-appdata-1 exited with code 1

If i disable oss i have nullpointer exeptions:

compose-init-appdata-1 | compose-init-appdata-1 | 2023-03-06 21:54:41.235 INFO 32 --- [ main] com.apitable.appdata.AppDataApplication : Starting AppDataApplication using Java 1.8.0_362 on cfeb6e92a0b7 with PID 32 (/app/init-appdata.jar started by root in /app) compose-init-appdata-1 | 2023-03-06 21:54:41.238 INFO 32 --- [ main] com.apitable.appdata.AppDataApplication : No active profile set, falling back to 1 default profile: "default" compose-init-appdata-1 | 2023-03-06 21:54:42.912 INFO 32 --- [ main] com.apitable.appdata.AppDataApplication : Started AppDataApplication in 2.224 seconds (JVM running for 2.692) compose-init-appdata-1 | java.lang.NullPointerException compose-init-appdata-1 | at com.apitable.appdata.loader.service.impl.AppDataLoaderServiceImpl.loadAsset(AppDataLoaderServiceImpl.java:75) compose-init-appdata-1 | at com.apitable.appdata.loader.service.impl.AppDataLoaderServiceImpl$$FastClassBySpringCGLIB$$65b15b21.invoke() compose-init-appdata-1 | at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) compose-init-appdata-1 | at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386) compose-init-appdata-1 | at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85) compose-init-appdata-1 | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:704) compose-init-appdata-1 | at com.apitable.appdata.loader.service.impl.AppDataLoaderServiceImpl$$EnhancerBySpringCGLIB$$b2e31686.loadAsset() compose-init-appdata-1 | at com.apitable.appdata.command.AdmCommand$Load.call(AdmCommand.java:52) compose-init-appdata-1 | at com.apitable.appdata.command.AdmCommand$Load.call(AdmCommand.java:43) compose-init-appdata-1 | at picocli.CommandLine.executeUserObject(CommandLine.java:2041) compose-init-appdata-1 | at picocli.CommandLine.access$1500(CommandLine.java:148) compose-init-appdata-1 | at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461) compose-init-appdata-1 | at picocli.CommandLine$RunLast.handle(CommandLine.java:2453) compose-init-appdata-1 | at picocli.CommandLine$RunLast.handle(CommandLine.java:2415) compose-init-appdata-1 | at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273) compose-init-appdata-1 | at picocli.CommandLine$RunLast.execute(CommandLine.java:2417) compose-init-appdata-1 | at picocli.CommandLine.execute(CommandLine.java:2170) compose-init-appdata-1 | at com.apitable.appdata.command.AdmCommandRunner.run(AdmCommandRunner.java:34) compose-init-appdata-1 | at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:768) compose-init-appdata-1 | at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:752) compose-init-appdata-1 | at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) compose-init-appdata-1 | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303) compose-init-appdata-1 | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292) compose-init-appdata-1 | at com.apitable.appdata.AppDataApplication.main(AppDataApplication.java:12) compose-init-appdata-1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) compose-init-appdata-1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) compose-init-appdata-1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) compose-init-appdata-1 | at java.lang.reflect.Method.invoke(Method.java:498) compose-init-appdata-1 | at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) compose-init-appdata-1 | at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) compose-init-appdata-1 | at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) compose-init-appdata-1 | at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) compose-init-appdata-1 exited with code 1

bytesnackscgn avatar Mar 06 '23 22:03 bytesnackscgn

@bytesnackscgn Sorry. The appdata service does not implement a variety of s3. It needs to rely on the environment variable values ​​of aws, such as AWS_ACCESS_KEY, AWS_ACCESS_SECRET, AWS_ENDPOINT, etc.

ChambersChan avatar Mar 07 '23 02:03 ChambersChan

Due to version iteration, each s3 implementation class may not implement all methods, so it is recommended to use compatible aws type configuration.

ChambersChan avatar Mar 07 '23 02:03 ChambersChan