nacos icon indicating copy to clipboard operation
nacos copied to clipboard

Server failed,Failed to obtain JDBC Connection;nested exception is java.sql.SQLTransientConnectionException:HikariPool-1-Connection is not available,request timed out after 3000ms

Open fiuty opened this issue 2 years ago • 8 comments

中文: 如下报错信息:

failed to request com.alibaba.nacos.api.exception.NacosException

Whitelabel Error Page

This application has no explicit mapping for /error, so you are seeing this as a fallback.

xxx CST 2022
There was an unexpected error (type=Internal Server Error, status=500).
Server failed,Failed to obtain JDBC Connection;nested exception is java.sql.SQLTransientConnectionException:HikariPool-1-Connection is not available,request timed out after 3000ms

以上报错信息是服务这边向nacos发送请求时的错误信息,偶发性会打印这样的日志,nacos连数据库是能正常访问的,数据库的cpu、io等指标也正常。 是部署nacos默认数据库连接池大小设置太小的原因导致吗?没改默认配置,看了配置文件默认的数据库连接数大小应该是20,是这个原因吗?这边服务数量是20多个,每个服务的实例是2个,相当于有40多个实例,频繁请求nacos发送心跳、查看配置等操作,导致数据库连接超时吗?

English: The following error message is reported: failed to request com.alibaba.nacos.api.exception.NacosException

Whitelabel Error Page

This application has no explicit mapping for /error, so you are seeing this as a fallback.

xxx CST 2022
There was an unexpected error (type=Internal Server Error, status=500).
Server failed,Failed to obtain JDBC Connection;nested exception is java.sql. SQLTransientConnectionException: HikariPool-1-Connection is not available, request timed out after 3000ms

The above error message is the error message when the service side sends a request to nacos. Occasionally, such a log will be printed. Nacos can access the database normally, and the cpu, io and other indicators of the database are also normal. Is it because the default database connection pool size setting of nacos is too small? I haven't changed the default configuration. After reading the configuration file, the default number of database connections should be 20. Is this the reason? The number of services here is more than 20, and each service has 2 instances, which is equivalent to more than 40 instances. Frequent requests for nacos to send heartbeats, view configurations, and other operations cause the database connection to time out?

fiuty avatar Sep 13 '22 15:09 fiuty

@realJackSun @KomachiSion

fiuty avatar Sep 13 '22 15:09 fiuty

看了下nacos数据库的连接数,也没满,默认配置是最大连接数20个,3个节点最大就60个连接数,但查了下nacos数据库连接数,当前才20多个连接数,也报该异常,3个节点都持有数据库的连接的,没占满

fiuty avatar Sep 14 '22 07:09 fiuty

request timed out after 3000ms

数据库访问超时了, 自查下网络/数据库是否有问题。

KomachiSion avatar Sep 15 '22 03:09 KomachiSion

或者是否有SQL执行过慢

KomachiSion avatar Sep 15 '22 03:09 KomachiSion

感谢回复,没有sql执行过慢,监控整个数据库,没有超过1秒的sql查询

fiuty avatar Sep 15 '22 03:09 fiuty

request timed out after 3000ms

数据库访问超时了, 自查下网络/数据库是否有问题。

3个节点我看都持久数据库的连接,nacos也正常在使用,但像服务端这边不断请求nacos比如发送心跳,时不时就会打印失败,报nacos数据库连接超时

fiuty avatar Sep 15 '22 03:09 fiuty

心跳不会走数据库,可以看下哪些SQL导致的数据库请求超时。

KomachiSion avatar Sep 19 '22 02:09 KomachiSion

1.目前看发送心跳也报错,然后返回错误信息也是HikariPool-1-Connection is not available,request timed out after 3000ms,除了发送心跳,其他请求也会报这个错,目前不会影响使用,推测部分请求有问题。 2.这边监控整个数据库,确实没有超过1秒的慢sql查询,像nacos这个报错,还能够通过查看哪些log来定位问题呢?

fiuty avatar Sep 19 '22 02:09 fiuty

  1. 发心跳不需要走数据库,所以我觉得你所说的心跳可能不是我理解的那个心跳
  2. 是不是网络有问题,无法及时收到回包? 如果数据库那边没有慢响应, 就arthas抓一下看看哪条sql执行超时了。

KomachiSion avatar Sep 23 '22 05:09 KomachiSion

No more response from author, and current information is not enough to find problem.

KomachiSion avatar Oct 10 '22 03:10 KomachiSion

推测是网络问题,近期基本没有该异常

fiuty avatar Oct 10 '22 03:10 fiuty