Getting false positive errors after failover

[amitgupta-ST]

Describe the bug

Hello, We are using an EJB3 based application that is deployed on JBoss EAP 6. We lookup on data-sources that connects to AWS Aurora DB cluster of 2 nodes. In order to make the application robust against DB failover, we are exploring the aws-labs MySQL driver. When we use this driver and we have a DB failover in the middle of some processing:

• The driver is able to connect to the new writer instance
• The DB read and writes are happening correctly (based on some data sampling done)

But our error rate goes very high as the driver starts reporting an error "Unable to establish SQL connection to writer node" for all the operations when there is a DB failover and the application is not restarted. Functionally this error seems to be false positive. But we wished to seek some assistance to figure out the cause and how to address this error too. Any pointer would be very helpful. Thanks!

Expected Behavior

There shouldn't be any error in the log if its connected to new write instance.

Current Behavior

• The driver is able to connect to the new writer instance
• The DB read and writes are happening correctly (based on some data sampling done)

But our error rate goes very high as the driver starts reporting an error "Unable to establish SQL connection to writer node" for all the operations when there is a DB failover and the application is not restarted. Functionally this error seems to be false positive. But we wished to seek some assistance to figure out the cause and how to address this error too.

Reproduction Steps

We have EJB based application which is deployed on Jboss EAP. All the connection details added in standalone.xml. We are using write node cluster url for connecting to database with aws driver.

<subsystem xmlns="urn:jboss:domain:datasources:1.1"> <datasources> <datasource jta="false" jndi-name="java:/jdbc/datasources/global" pool-name="GlobalDS" enabled="true" use-ccm="false"> <connection-url>jdbc:mysql:aws://dev-xcloud-aurora-cluster.cluster-ceq12wejj76d.us-east-1.rds.amazonaws.com:3306?useSSL=false</connection-url> <driver-class>software.aws.rds.jdbc.mysql.Driver</driver-class> <driver>mysql</driver> <pool> <min-pool-size>25</min-pool-size> <max-pool-size>50</max-pool-size> <flush-strategy>IdleConnections</flush-strategy> </pool> <security> <user-name></user-name> <password></password> </security> <validation> <check-valid-connection-sql>SELECT 1</check-valid-connection-sql> <background-validation>true</background-validation> </validation> <statement> <share-prepared-statements>false</share-prepared-statements> </statement> </datasource> <drivers> <driver name="mysql" module="com.mysql"> <xa-datasource-class>com.mysql.jdbc.jdbc2.optional.MysqlXADataSource</xa-datasource-class> </driver> <driver name="h2" module="com.h2database.h2"> <xa-datasource-class>org.h2.jdbcx.JdbcDataSource</xa-datasource-class> </driver> </drivers> </datasources> </subsystem>

Possible Solution

No response

Additional Information/Context

No response

The AWS JDBC Driver for MySQL version used

1.0

JDK version used

1.8

Operating System and version

EC2 instance of AWS

[sergiyvamz]

Hello @amitgupta-ST

Thank you for reaching out! Would you mind to provide a bit more details about the issue you're experiencing? That'd help us to investigate the issue.

• Can you provide driver log for a "Unable to establish SQL connection to writer node" event? Driver logging can be turned on by adding a configuration parameter to a connection string. More details can be found here https://github.com/awslabs/aws-mysql-jdbc#enable-logging
• There's a snapshot build available and there's a big chance that the issue that you're reporting is already fixed in that build. Can you try to repro the issue with a snapshot build? https://github.com/awslabs/aws-mysql-jdbc#using-a-snapshot-of-the-driver
• Do you use any connection pool in your EJB3 application?
• Can you provide more details about usual SQL statements and usual SQL execution time in your EJB3 application? Does your application execute simple SELECT statements with short execution time (up to few seconds)? Does it mostly execute long running SQL statements (execution time in minutes and longer)? Most of SQL statements are DML statements?

Thank you!