Open
abedali
opened this issue 3 years ago
•
20 comments
Please help, we need this for our feature enhancements.
Spring boot
aws-jdbc-driver 1.1.0
Hikari2.7.9
AWSSecretsManagerPluginFactory
SecretsId is submitted through HikariConfig dataSourceProperties
HikariConfig - dataSourceProperties............{secretsManagerSecretId=$validSecretIsd$, password=, secretsManagerRegion=us-east-1}
Refer to the following stack trace.
JDBC Connectivity is failing.
HikariConfig properties are in the following log output.
Please refer to 'jdbcUrl' & 'dataSourceProperties'
Looks like a problem in reading the secrets. Creds are empty
Error exception
169 2022-06-30T20:37:33.164-04:00 Caused by: software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.CJException: Access denied for user ''@'172.22.5.247' (using password: NO)
2022-06-30T20:37:33.191-04:00 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
2022-06-30T20:37:33.191-04:00 at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1105)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:307)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1778)
2022-06-30T20:37:33.191-04:00 org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.DatabaseException: java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
2022-06-30T20:37:33.191-04:00 30-06-2022 20:37:33.190 main [ERROR] SpringApplication - Application run failed
2022-06-30T20:37:33.191-04:00 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2022-06-30T20:37:33.191-04:00 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2022-06-30T20:37:33.191-04:00 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2022-06-30T20:37:33.191-04:00 Caused by: software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.CJException: Access denied for user ''@'172.22.5.247' (using password: NO)
2022-06-30T20:37:33.191-04:00 ... 28 common frames omitted
2022-06-30T20:37:33.191-04:00 at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:266)
2022-06-30T20:37:33.191-04:00 at com.zaxxer.hikari.HikariDataSource$$EnhancerBySpringCGLIB$$d17dd0d2.getConnection()
2022-06-30T20:37:33.191-04:00 at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
2022-06-30T20:37:33.191-04:00 at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
2022-06-30T20:37:33.191-04:00 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
2022-06-30T20:37:33.191-04:00 at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
2022-06-30T20:37:33.191-04:00 at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
2022-06-30T20:37:33.191-04:00 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
2022-06-30T20:37:33.191-04:00 at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
2022-06-30T20:37:33.191-04:00 at com.zaxxer.hikari.HikariDataSource$$FastClassBySpringCGLIB$$eeb1ae86.invoke()
2022-06-30T20:37:33.191-04:00 at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
2022-06-30T20:37:33.191-04:00 at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:115)
2022-06-30T20:37:33.191-04:00 at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:534)
2022-06-30T20:37:33.191-04:00 at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:460)
2022-06-30T20:37:33.191-04:00 at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:194)
2022-06-30T20:37:33.191-04:00 at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:365)
2022-06-30T20:37:33.191-04:00 at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:117)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:221)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.ConnectionProxy.autodetectClusterAndCreateProxyInstance(ConnectionProxy.java:126)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.ConnectionProxy.(ConnectionProxy.java:91)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.ConnectionProxy.(ConnectionProxy.java:79)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.ConnectionProxy.(ConnectionProxy.java:104)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.ConnectionProxy.initPluginManager(ConnectionProxy.java:283)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.DefaultConnectionPlugin.openInitialConnection(DefaultConnectionPlugin.java:104)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.BasicConnectionProvider.connect(BasicConnectionProvider.java:53)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:253)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.failover.FailoverConnectionPlugin.openInitialConnection(FailoverConnectionPlugin.java:219)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.failover.FailoverConnectionPlugin.createConnection(FailoverConnectionPlugin.java:1060)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.NodeMonitoringConnectionPlugin.openInitialConnection(NodeMonitoringConnectionPlugin.java:255)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.AWSSecretsManagerPlugin.openInitialConnection(AWSSecretsManagerPlugin.java:140)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.AWSSecretsManagerPlugin.attemptConnectionWithSecrets(AWSSecretsManagerPlugin.java:170)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.ConnectionPluginManager.openInitialConnection(ConnectionPluginManager.java:151)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:73)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:910)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:460)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:835)
2022-06-30T20:37:33.191-04:00 Caused by: liquibase.exception.DatabaseException: java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
2022-06-30T20:37:33.191-04:00 at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1837)
2022-06-30T20:37:33.191-04:00 at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:271)
2022-06-30T20:37:33.191-04:00 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1774)
2022-06-30T20:37:33.191-04:00 Caused by: java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
2022-06-30T20:37:33.191-04:00 ... 26 common frames omitted
2022-06-30T20:37:33.191-04:00 at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
2022-06-30T20:37:33.191-04:00 at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
2022-06-30T20:37:33.191-04:00 at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
2022-06-30T20:37:33.191-04:00 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-06-30T20:37:33.191-04:00 at java.lang.reflect.Method.invoke(Method.java:498)
2022-06-30T20:37:33.191-04:00 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2022-06-30T20:37:33.191-04:00 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-06-30T20:37:33.191-04:00 at com.dt.rts.eregusa.user.LoginApplication.main(LoginApplication.java:58)
2022-06-30T20:37:33.191-04:00 at org.springframework.boot.SpringApplication.run(SpringApplication.java:1204)
2022-06-30T20:37:33.191-04:00 at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
2022-06-30T20:37:33.191-04:00 at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
2022-06-30T20:37:33.191-04:00 at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
2022-06-30T20:37:33.191-04:00 at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744)
2022-06-30T20:37:33.191-04:00 at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
2022-06-30T20:37:33.191-04:00 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
2022-06-30T20:37:33.191-04:00 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:709)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:787)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:129)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:854)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.NativeSession.connect(NativeSession.java:136)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1369)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:216)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeAuthenticationProvider.proceedHandshakeWithPluggableAuthentication(NativeAuthenticationProvider.java:509)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:137)
2022-06-30T20:37:33.191-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:677)
2022-06-30T20:37:33.191-04:00 ... 62 common frames omitted
2022-06-30T20:37:33.188-04:00 Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-06-30T20:37:33.188-04:00 2022-06-30T20:37:33.188-04:00
30-06-2022 20:37:33.188 main [INFO ] ConditionEvaluationReportLoggingListener -
2022-06-30T20:37:33.168-04:00 30-06-2022 20:37:33.168 main [INFO ] DirectJDKLog - Stopping service [Tomcat]
2022-06-30T20:37:33.165-04:00 30-06-2022 20:37:33.165 main [WARN ] AbstractApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource [org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: Invocation of init method failed; nested exception is liquibase.exception.DatabaseException: java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
2022-06-30T20:37:33.164-04:00 30-06-2022 20:37:33.163 main [ERROR] HikariPool - LOGINHikariPool - Exception during pool initialization.
2022-06-30T20:37:33.164-04:00 java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:835)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:910)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:73)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.ConnectionProxy.(ConnectionProxy.java:79)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.ConnectionProxy.(ConnectionProxy.java:91)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.ConnectionProxy.(ConnectionProxy.java:104)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.ConnectionProxy.initPluginManager(ConnectionProxy.java:283)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.ConnectionPluginManager.openInitialConnection(ConnectionPluginManager.java:151)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.failover.FailoverConnectionPlugin.openInitialConnection(FailoverConnectionPlugin.java:219)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.failover.FailoverConnectionPlugin.createConnection(FailoverConnectionPlugin.java:1060)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.NodeMonitoringConnectionPlugin.openInitialConnection(NodeMonitoringConnectionPlugin.java:255)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.AWSSecretsManagerPlugin.openInitialConnection(AWSSecretsManagerPlugin.java:140)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.AWSSecretsManagerPlugin.attemptConnectionWithSecrets(AWSSecretsManagerPlugin.java:170)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.DefaultConnectionPlugin.openInitialConnection(DefaultConnectionPlugin.java:104)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.BasicConnectionProvider.connect(BasicConnectionProvider.java:53)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:253)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:460)
2022-06-30T20:37:33.164-04:00 at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:534)
2022-06-30T20:37:33.164-04:00 at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:460)
2022-06-30T20:37:33.164-04:00 at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:194)
2022-06-30T20:37:33.164-04:00 at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:365)
2022-06-30T20:37:33.164-04:00 at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:117)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:221)
2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.ConnectionProxy.autodetectClusterAndCreateProxyInstance(ConnectionProxy.java:126)
2022-06-30T20:37:33.164-04:00 at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
2022-06-30T20:37:33.164-04:00 at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:115)
2022-06-30T20:37:33.164-04:00 at com.zaxxer.hikari.HikariDataSource$$FastClassBySpringCGLIB$$eeb1ae86.invoke()
2022-06-30T20:37:33.164-04:00 at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
2022-06-30T20:37:33.164-04:00 at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
2022-06-30T20:37:33.164-04:00 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
2022-06-30T20:37:33.164-04:00 at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
2022-06-30T20:37:33.164-04:00 at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
2022-06-30T20:37:33.164-04:00 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
2022-06-30T20:37:33.164-04:00 at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
2022-06-30T20:37:33.164-04:00 at com.zaxxer.hikari.HikariDataSource$$EnhancerBySpringCGLIB$$d17dd0d2.getConnection()
2022-06-30T20:37:33.164-04:00 at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:266)
2022-06-30T20:37:33.164-04:00 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1837)
2022-06-30T20:37:33.164-04:00 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1774)
2022-06-30T20:37:33.164-04:00 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
2022-06-30T20:37:33.164-04:00 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
2022-06-30T20:37:33.164-04:00 at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
2022-06-30T20:37:33.164-04:00 at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
2022-06-30T20:37:33.164-04:00 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
2022-06-30T20:37:33.164-04:00 at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:307)
151 2022-06-30T20:37:33.164-04:00 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
152 2022-06-30T20:37:33.164-04:00 at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1105)
153 2022-06-30T20:37:33.164-04:00 at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
154 2022-06-30T20:37:33.164-04:00 at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
155 2022-06-30T20:37:33.164-04:00 at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
156 2022-06-30T20:37:33.164-04:00 at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
157 2022-06-30T20:37:33.164-04:00 at com.dt.rts.eregusa.user.LoginApplication.main(LoginApplication.java:58)
158 2022-06-30T20:37:33.164-04:00 at org.springframework.boot.SpringApplication.run(SpringApplication.java:1204)
159 2022-06-30T20:37:33.164-04:00 at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
160 2022-06-30T20:37:33.164-04:00 at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
161 2022-06-30T20:37:33.164-04:00 at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
162 2022-06-30T20:37:33.164-04:00 at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744)
163 2022-06-30T20:37:33.164-04:00 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
164 2022-06-30T20:37:33.164-04:00 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
165 2022-06-30T20:37:33.164-04:00 at java.lang.reflect.Method.invoke(Method.java:498)
166 2022-06-30T20:37:33.164-04:00 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
167 2022-06-30T20:37:33.164-04:00 at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
168 2022-06-30T20:37:33.164-04:00 at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
169 2022-06-30T20:37:33.164-04:00 Caused by: software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.CJException: Access denied for user ''@'172.22.5.247' (using password: NO)
170 2022-06-30T20:37:33.164-04:00 at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
171 2022-06-30T20:37:33.164-04:00 at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
172 2022-06-30T20:37:33.164-04:00 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
173 2022-06-30T20:37:33.164-04:00 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
174 2022-06-30T20:37:33.164-04:00 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
175 2022-06-30T20:37:33.164-04:00 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
176 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
177 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:709)
178 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:787)
179 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:129)
180 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
181 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
182 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:216)
183 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeAuthenticationProvider.proceedHandshakeWithPluggableAuthentication(NativeAuthenticationProvider.java:509)
184 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:137)
185 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:677)
186 2022-06-30T20:37:33.164-04:00 .. 62 common frames omitted
187 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:854)
188 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.NativeSession.connect(NativeSession.java:136)
189 2022-06-30T20:37:33.164-04:00 at software.aws.rds.jdbc.mysql.shading.com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1369)
190 2022-06-30T20:37:27.414-04:00 30-06-2022 20:37:27.414 main [INFO ] HikariDataSource - LOGINHikariPool - Starting...
191 2022-06-30T20:37:27.414-04:00 30-06-2022 20:37:27.414 main [DEBUG] HikariConfig - validationTimeout...............30000
192 2022-06-30T20:37:27.414-04:00 30-06-2022 20:37:27.413 main [DEBUG] HikariConfig - username........................none
193 2022-06-30T20:37:27.413-04:00 30-06-2022 20:37:27.413 main [DEBUG] HikariConfig - transactionIsolation............default
194 2022-06-30T20:37:27.413-04:00 30-06-2022 20:37:27.413 main [DEBUG] HikariConfig - threadFactory...................internal
195 2022-06-30T20:37:27.413-04:00 30-06-2022 20:37:27.413 main [DEBUG] HikariConfig - schema..........................none
196 2022-06-30T20:37:27.413-04:00 30-06-2022 20:37:27.413 main [DEBUG] HikariConfig - scheduledExecutorService........internal
197 2022-06-30T20:37:27.413-04:00 30-06-2022 20:37:27.413 main [DEBUG] HikariConfig - scheduledExecutor...............none
198 2022-06-30T20:37:27.413-04:00 30-06-2022 20:37:27.413 main [DEBUG] HikariConfig - registerMbeans..................true
199 2022-06-30T20:37:27.413-04:00 30-06-2022 20:37:27.412 main [DEBUG] HikariConfig - readOnly........................false
200 2022-06-30T20:37:27.412-04:00 30-06-2022 20:37:27.412 main [DEBUG] HikariConfig - poolName........................"LOGINHikariPool"
201 2022-06-30T20:37:27.412-04:00 30-06-2022 20:37:27.412 main [DEBUG] HikariConfig - password........................
202 2022-06-30T20:37:27.412-04:00 30-06-2022 20:37:27.412 main [DEBUG] HikariConfig - minimumIdle.....................5
203 2022-06-30T20:37:27.412-04:00 30-06-2022 20:37:27.412 main [DEBUG] HikariConfig - metricsTrackerFactory...........none
204 2022-06-30T20:37:27.412-04:00 30-06-2022 20:37:27.412 main [DEBUG] HikariConfig - metricRegistry..................none
205 2022-06-30T20:37:27.412-04:00 30-06-2022 20:37:27.411 main [DEBUG] HikariConfig - maximumPoolSize.................25
206 2022-06-30T20:37:27.411-04:00 30-06-2022 20:37:27.411 main [DEBUG] HikariConfig - maxLifetime.....................120000
207 2022-06-30T20:37:27.411-04:00 30-06-2022 20:37:27.411 main [DEBUG] HikariConfig - leakDetectionThreshold..........0
208 2022-06-30T20:37:27.411-04:00 30-06-2022 20:37:27.411 main [DEBUG] HikariConfig - jdbcUrl.........................jdbc:mysql:aws://<RDS_Cluster>:3306/<DBNAME>?maxAllowedPacket=99999744&socketTimeout=30000&autoReconnect=true&serverTimezone=UTC&enabledTLSProtocols=TLSv1.2&connectionPluginFactories=software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.failover.FailoverConnectionPluginFactory,software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.NodeMonitoringConnectionPluginFactory, software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.AWSSecretsManagerPluginFactory
209 2022-06-30T20:37:27.411-04:00 30-06-2022 20:37:27.411 main [DEBUG] HikariConfig - jdbc4ConnectionTest.............false
210 2022-06-30T20:37:27.411-04:00 30-06-2022 20:37:27.411 main [DEBUG] HikariConfig - isolateInternalQueries..........false
211 2022-06-30T20:37:27.411-04:00 30-06-2022 20:37:27.410 main [DEBUG] HikariConfig - initializationFailTimeout.......1
212 2022-06-30T20:37:27.410-04:00 30-06-2022 20:37:27.410 main [DEBUG] HikariConfig - initializationFailFast..........true
213 2022-06-30T20:37:27.410-04:00 30-06-2022 20:37:27.410 main [DEBUG] HikariConfig - idleTimeout.....................0
214 2022-06-30T20:37:27.410-04:00 30-06-2022 20:37:27.410 main [DEBUG] HikariConfig - healthCheckRegistry.............none
215 2022-06-30T20:37:27.410-04:00 30-06-2022 20:37:27.410 main [DEBUG] HikariConfig - healthCheckProperties...........{}
216 2022-06-30T20:37:27.410-04:00 30-06-2022 20:37:27.410 main [DEBUG] HikariConfig - driverClassName................."software.aws.rds.jdbc.mysql.Driver"
217 2022-06-30T20:37:27.410-04:00 30-06-2022 20:37:27.409 main [DEBUG] HikariConfig - dataSourceProperties............{secretsManagerSecretId=$validSecretsId$, password=, secretsManagerRegion=us-east-1}
218 2022-06-30T20:37:27.409-04:00 30-06-2022 20:37:27.409 main [DEBUG] HikariConfig - dataSourceJNDI..................none
219 2022-06-30T20:37:27.409-04:00 30-06-2022 20:37:27.409 main [DEBUG] HikariConfig - dataSourceClassName.............none
220 2022-06-30T20:37:27.409-04:00 30-06-2022 20:37:27.408 main [DEBUG] HikariConfig - dataSource......................none
221 2022-06-30T20:37:27.408-04:00 30-06-2022 20:37:27.408 main [DEBUG] HikariConfig - connectionTimeout...............60000
222 2022-06-30T20:37:27.408-04:00 30-06-2022 20:37:27.408 main [DEBUG] HikariConfig - connectionTestQuery............."SELECT 1 FROM DUAL"
223 2022-06-30T20:37:27.408-04:00 30-06-2022 20:37:27.408 main [DEBUG] HikariConfig - connectionInitSql...............none
224 2022-06-30T20:37:27.408-04:00 30-06-2022 20:37:27.408 main [DEBUG] HikariConfig - catalog.........................none
225 2022-06-30T20:37:27.408-04:00 30-06-2022 20:37:27.408 main [DEBUG] HikariConfig - autoCommit......................true
226 2022-06-30T20:37:27.408-04:00 30-06-2022 20:37:27.407 main [DEBUG] HikariConfig - allowPoolSuspension.............true
Thank you for reaching out. As I can see in provided logs, "secretsManagerSecretId" parameter is blank. Did you just remove it from logs for security reasons? Please confirm. It's essential to pass this parameter to the driver.
AWS Secret Manager also requires valid AWS credentials. Please make sure that AWS credentials are properly configured. More details could be found at https://docs.aws.amazon.com/sdkref/latest/guide/creds-config-files.html
@sergiyvamz thanks for your response, secretsManagerSecretId value is purposely removed from the post.
The username and password stored in the secrets are valid and I tested by reading them from secrets using aws SDK and JDBC connectivity worked. This means that my Docker container has the right configuration to read credentials from secrets.
The problem is only when I replace credentials with secretsId.
There is a difference I see is, I'm using 'com.amazonaws:aws-java-sdk-secretsmanager' for reading creds from secrets,
Where as aws-jdbc-driver uses 'software.amazon.awssdk:secretsmanager' library.
@abedali, thanks for providing some additional detail. The difference you see between com.amazonaws:aws-java-sdk-secretsmanager and software.amazon.awssdk:secretsmanager is that software.amazon.awssdk:secretsmanager uses version 2 of the Java SDK whereas the first one you listed uses version 1 of the Java SDK.
Are you able to retest your workflow without the driver using version 2 of the Java SDK? Also, you mentioned replacing your credentials with secretsId. If you're using the sample code as a reference, this value ought to be the actual secret name or ARN. Would you be able to try both?
@hsuamz thanks for the response.
Your question-1: Are you able to retest your workflow without the driver using Java SDK V2?
Answer: Yes, (Without V2 Java SDK secretsmanager) + aws jdbc driver 1.1.0 + Hikari 2.7.9 + AWSSecretsManagerPluginFactory
Application is throwing an exception
"java.lang.ClassNotFoundException: software.amazon.awssdk.services.secretsmanager.model.SecretsManagerException"
Your question-2: Secret Name (or) Secret ARN
Answer: I tried passing both to the SecretsManager Plugin, didn't work. Same exact Stacktrace that I pasted above.
My application is able to read secrets using V1 & V2 Java sdk (both using secret name & secret ARN).
Question: HikariConfig dataSourceProperties attribute that has following key:value pair. Is this being propagated to Connection object for establishing the connection?
secretsManagerSecretId & secretsManagerRegion are custom properties to Hikari.
This exception means that Secret Manager plugin can't find AWS SDK classes that it depends on. Please make sure that all required AWS SDK v.2 libraries and their dependencies are properly mentioned in classpath and your application can load them.
@sergiyvamz Thanks for the response
The error stackTrace in my original post is after I included 'software.amazon.awssdk:secretsmanager:2.17.191' in the classpath. I'm sorry if I've missed mentioning this in my original post.
I've responded with that error message because @hsuamz asked a question.
Just to avoid going in circles, please feel free to schedule a call or please guide me on how to arrange one, so that we can discuss it.
@sergiyvamz not anymore after including 'software.amazon.awssdk:secretsmanager:2.17.191' in the classpath.
The problem I'm experiencing is pasted in my original post.
Its a different exception, seems like plugin couldn't read credentials from secrets (or) Secret.java POJO is instantiated way before the secrets are read.
@sergiyvamz thanks for the response.
May be you are not reading my response completely.
My original post has the stacktrace from Day-1(since I reported the issue) and it is after including 'software.amazon.awssdk:secretsmanager:2.17.191' library in the classpath.
Hi @abedali, can you please ensure the AWSSecretsManagerPlugin is the first in the plugin chain by specifying the connectionPluginFactories parameter as follows:
I've updated plugins order in connectionPluginFactories attribute and tested. The issue still exists. I've attached the error stacktrace to this message (stacktrace is too big to paste here).
One observation in the following log statement, password is masked(though I didn't submit password) may be Hikari is defaulting to empty password if application doesn't supply one and the same might have shared with aws jdbc driver, as a result driver might be assuming that its credentials based connection as supposed to Secrets.
BYW, we are using Hikari 2.7.9 (this version recovers from az fail-over faster).
07-07-2022 16:09:55.923 main [DEBUG] HikariConfig - dataSourceProperties............{secretsManagerSecretId=np-cert/login/aurora, password=, secretsManagerRegion=us-east-1}
I've been able to confirm that the following code without Hikari works. Can you test it please?
import java.util.Properties;
import software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.AWSSecretsManagerPluginFactory;
public class TestSecretsManager {
private static final String CONNECTION_STRING = "jdbc:mysql:aws://<changeme>:3306/mysql";
private static final String SECRET_ID = <yoursecretid> //"secretId";
private static final String REGION = <yourregion> //"us-east-1";
public static void main(String[] args) throws SQLException {
final Properties properties = new Properties();
// Enable the AWS Secrets Manager Plugin:
properties.setProperty("connectionPluginFactories", AWSSecretsManagerPluginFactory.class.getName());
// Set these properties so secrets can be retrieved:
properties.setProperty("secretsManagerSecretId", SECRET_ID);
properties.setProperty("secretsManagerRegion", REGION);
// Try and make a connection:
try (final Connection conn = DriverManager.getConnection(CONNECTION_STRING, properties);
final Statement statement = conn.createStatement();
final ResultSet rs = statement.executeQuery("SELECT 1")) {
while (rs.next()) {
System.out.println( "Result is : " + rs.getInt(1));
}
}
}
}
@davecramer
I'm able to reproduce the problem by modifying your standalone program
The culprit is 'autoReconnect=true' in the Connection URL
If I place 'autoReconnect=true' property in the connection url, then I get the same error
Access denied for user ''@'IP_ADDRESS_MASKED' (using password: NO)
Not sure about the significance of 'autoReconnect=true' in jdbc connection through secrets.
I tried in my spring boot application as well, by deleting 'autoReconnect=true', JDBC Connectivity through Hikari 2.7.9 Worked.
I confirm that 'autoReconnect=true' is the culprit
There's a new snapshot available. It contains some improvements for Secret Manager plugin. Will you be able to check it and confirm your issue is fixed? Thank you!
https://github.com/awslabs/aws-mysql-jdbc#using-a-snapshot-of-the-driver