aws-postgresql-jdbc icon indicating copy to clipboard operation
aws-postgresql-jdbc copied to clipboard

AWS RDS Postgresql error - marked as broken because of SQLSTATE(08006), ErrorCode(0)

Open playnoob96 opened this issue 3 years ago • 2 comments

Please read https://stackoverflow.com/help/minimal-reproducible-example

Describe the issue This error pops up after AWS RDS Auto Scaling is enabled and when the added read replicas are getting removed when the DB Cluster is scaling in.

HikariPool-1 - Connection connection name marked as broken because of SQLSTATE(08006), ErrorCode(0)

]; An I/O error occurred while sending to the backend.; nested exception

Driver Version? 0.1.0

Java Version? 8

OS Version? Linux

PostgreSQL Version? 9.6.19

To Reproduce Enable Auto Scaling, when the replicas are scaling in can observe the issue

Expected behavior Incoming requests should not fail while scaling in

playnoob96 avatar Aug 25 '21 17:08 playnoob96

Thank you for bringing up this issue. We've checked the mentioned scenario, but couldn't reproduce this. In our investigation, the driver reconnected to other available nodes while a cluster was scaling in. Such behavior is an expected one. We'd like to continue our investigation and find the root cause of the issue that you mentioned. Would you mind clarifying some details?

  • What cluster endpoint name was used to open the connection? Cluster endpoint, cluster reader endpoint, or instance endpoint?
  • Was it a read-only connection?
  • What was the cluster configuration/topology at the moment when the issue occurred? How many readers did it have? Including how many auto-scaled readers?
  • If possible, please provide a full stack trace of the exception.
  • If possible, please provide the finest driver logs.

The mentioned issue doesn't look like expected driver behavior and it definitely should be further investigated. However, we'd like to mention that connection failures are an everyday situation and the application is supposed to handle them. Besides DB cluster failures there are possible network outages of different kinds and they may lead to broken communication and invalid connections. The driver may not be able to successfully reconnect to other instances in such cases.

sergiyvamz avatar Aug 31 '21 18:08 sergiyvamz

For visibility, we are recommending everyone to move to our new repository here: https://github.com/awslabs/aws-advanced-jdbc-wrapper.

We have redesigned the driver as a JDBC Wrapper which has been tested to work with the PGJDBC driver. Please give it a look and we will be triaging issues in the new repository.

Thanks.

hsuamz avatar Oct 06 '22 18:10 hsuamz