Connection Timeout with Hikari and Mysql

[NishanthShajahan]

Hello, We are seeing the below exceptions few times a day and there hasn't been a pattern to this as far as I can see. Some of the requests that come into the server gets errored out due to this while the other requests that come in exactly at the same time is successful.

Hikari - 4.0.3 Mysql - 5.7

An error occurred while attempting to access the database. at com.company.dbconnectors.exception.DbException.causedBy(DbException.java:86) at com.company.dbconnectors.db.DatabaseBase.openAutocommitConnection(DatabaseBase.java:81) at com.company.dbconnectors.db.DatabaseBase.autocommit(DatabaseBase.java:148) at com.company.dbconnectors.transactor.DatabaseTransactor.autocommitRO(DatabaseTransactor.java:76) at com.company.dbconnectors.transactor.DatabaseTransactor.selectStringsRO(DatabaseTransactor.java:195) at com.company.security.token.AuthTokenURIProviderImpl.getUris(AuthTokenURIProviderImpl.java:28) at com.company.security.service.DefaultAuthenticationService.getURIsForToken(DefaultAuthenticationService.java:169) at com.company.security.token.UriAuthAuthority.uriTokenIsValid(UriAuthAuthority.java:87) at com.company.security.token.UriAuthAuthority.authenticate(UriAuthAuthority.java:48) at com.company.security.token.UriAuthAuthority.authenticate(UriAuthAuthority.java:43) at com.company.security.authority.CompositeAuthenticationAuthority.authenticate(CompositeAuthenticationAuthority.java:47) at com.company.billing.v1.security.AuthenticationAuthority.authenticate(AuthenticationAuthority.java:33) at com.company.security.handler.AuthenticationHandler.tryAuthenticate(AuthenticationHandler.java:147) at com.company.security.handler.AuthenticationHandler.access$300(AuthenticationHandler.java:49) at com.company.security.handler.AuthenticationHandler$AuthenticationRunnable.doRun(AuthenticationHandler.java:121) at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) at io.netty.util.concurrent.PromiseTask.run(PromiseTask.java:106) at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.SQLTransientConnectionException: meta.r - Connection is not available, request timed out after 4006ms. at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696) at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:197) at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:162) at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:100) at com.company.dbconnectors.db.DatabaseBase.openAutocommitConnection(DatabaseBase.java:64) ... 20 more Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:404) at com.mysql.jdbc.Util.getInstance(Util.java:387) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:917) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:896) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:885) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860) at com.mysql.jdbc.ConnectionImpl.throwConnectionClosedException(ConnectionImpl.java:1246) at com.mysql.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:1241) at com.mysql.jdbc.ConnectionImpl.setNetworkTimeout(ConnectionImpl.java:5521) at com.zaxxer.hikari.pool.PoolBase.setNetworkTimeout(PoolBase.java:566) at com.zaxxer.hikari.pool.PoolBase.isConnectionAlive(PoolBase.java:173) at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:186)