polardbx-sql icon indicating copy to clipboard operation
polardbx-sql copied to clipboard

Published 20 hours ago • verified publisher icon polardb

galaxysql process crashed during sysbench tests

Open wangdong58 opened this issue 4 years ago • 2 comments

I deployed a PolarDB-X clusters using the PXD tool，the galaxysql process crashed during sysbench tests.

/home/admin/drds-server/logs/tddl/XLog.log

2021-10-28 17:14:00.403 ERROR ERR-CODE: [PXC-10004][ERR_X_PROTOCOL_RESULT] Fatal error when fetch data: Expectation failed: no_error HY000 com.alibaba.polardbx.common.exception.TddlRuntimeException: ERR-CODE: [PXC-10004][ERR_X_PROTOCOL_RESULT] Fatal error when fetch data: Expectation failed: no_error HY000 at com.alibaba.polardbx.rpc.result.XResult.internalFetchOneObject(XResult.java:921) at com.alibaba.polardbx.rpc.result.XResult.waitFinish(XResult.java:508) at com.alibaba.polardbx.rpc.client.XSession.flushIgnorable(XSession.java:889) at com.alibaba.polardbx.rpc.pool.XConnection.close(XConnection.java:112) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.initTsoServicesX(MetaDbDataSource.java:132) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.initXDataSourceByJdbcProps(MetaDbDataSource.java:170) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.initMetaDbDataSource(MetaDbDataSource.java:181) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.doInit(MetaDbDataSource.java:121) at com.alibaba.polardbx.common.model.lifecycle.AbstractLifecycle.init(AbstractLifecycle.java:43) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.getInstance(MetaDbDataSource.java:462) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.initMetaDbDataSource(MetaDbDataSource.java:488) at com.alibaba.polardbx.config.loader.ServerLoader.initPolarDbXComponents(ServerLoader.java:207) at com.alibaba.polardbx.config.loader.ServerLoader.load(ServerLoader.java:181) at com.alibaba.polardbx.config.loader.ServerLoader.doInit(ServerLoader.java:91) at com.alibaba.polardbx.common.model.lifecycle.AbstractLifecycle.init(AbstractLifecycle.java:43) at com.alibaba.polardbx.CobarConfig.initCobarConfig(CobarConfig.java:68) at com.alibaba.polardbx.CobarConfig.(CobarConfig.java:61) at com.alibaba.polardbx.CobarServer.(CobarServer.java:141) at com.alibaba.polardbx.CobarServer.(CobarServer.java:95) at com.alibaba.polardbx.server.TddlLauncher.main(TddlLauncher.java:82)

wangdong58 avatar Oct 28 '21 09:10 wangdong58

这个看起来像请求中间的no_error的Expectation违反了（这个是请求流水线的一个机制），查看下对应的库 /home/admin/drds-server/logs/库名/tddl.log 有没有其他同时间的报错信息

chenyu-zzy avatar Oct 29 '21 08:10 chenyu-zzy

1、又测试了一次，/home/admin/drds-server/logs/tddl/XLog.log如下： 2021-11-02 14:06:22.120 ERROR ERR-CODE: [PXC-10004][ERR_X_PROTOCOL_RESULT] Fatal error when fetch data: Expectation failed: no_error HY000 com.alibaba.polardbx.common.exception.TddlRuntimeException: ERR-CODE: [PXC-10004][ERR_X_PROTOCOL_RESULT] Fatal error when fetch data: Expectation failed: no_error HY000 at com.alibaba.polardbx.rpc.result.XResult.internalFetchOneObject(XResult.java:921) at com.alibaba.polardbx.rpc.result.XResult.waitFinish(XResult.java:508) at com.alibaba.polardbx.rpc.client.XSession.flushIgnorable(XSession.java:889) at com.alibaba.polardbx.rpc.pool.XConnection.close(XConnection.java:112) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.initTsoServicesX(MetaDbDataSource.java:132) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.initXDataSourceByJdbcProps(MetaDbDataSource.java:170) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.initMetaDbDataSource(MetaDbDataSource.java:181) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.doInit(MetaDbDataSource.java:121) at com.alibaba.polardbx.common.model.lifecycle.AbstractLifecycle.init(AbstractLifecycle.java:43) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.getInstance(MetaDbDataSource.java:462) at com.alibaba.polardbx.gms.metadb.MetaDbDataSource.initMetaDbDataSource(MetaDbDataSource.java:488) at com.alibaba.polardbx.config.loader.ServerLoader.initPolarDbXComponents(ServerLoader.java:207) at com.alibaba.polardbx.config.loader.ServerLoader.load(ServerLoader.java:181) at com.alibaba.polardbx.config.loader.ServerLoader.doInit(ServerLoader.java:91) at com.alibaba.polardbx.common.model.lifecycle.AbstractLifecycle.init(AbstractLifecycle.java:43) at com.alibaba.polardbx.CobarConfig.initCobarConfig(CobarConfig.java:68) at com.alibaba.polardbx.CobarConfig.(CobarConfig.java:61) at com.alibaba.polardbx.CobarServer.(CobarServer.java:141) at com.alibaba.polardbx.CobarServer.(CobarServer.java:95) at com.alibaba.polardbx.server.TddlLauncher.main(TddlLauncher.java:82)

2、对应数据库sbtestdb下的日志/home/admin/drds-server/logs/sbtestdb/tddl.log 2021-11-02 14:01:19.344 [ServerExecutor-bucket-1-18-thread-9] WARN com.alibaba.polardbx.optimizer.planmanager.PlanManager - [user=polardbx_root,host=,port=50370,schema=sbtestdb] [TDDL] PlanManager init consuming 0.011 seconds, tddl version: 5.4.12-20211018 2021-11-02 14:19:32.138 [ServerExecutor-bucket-1-18-thread-2] WARN com.alibaba.polardbx.optimizer.planmanager.PlanManager - [user=polardbx_root,host=,port=55074,schema=sbtestdb] [TDDL] PlanManager init consuming 0.012 seconds, tddl version: 5.4.12-20211018 2021-11-02 14:19:42.617 [ServerExecutor-bucket-1-18-thread-6] ERROR com.alibaba.polardbx.net.handler.FrontendCommandHandler - [user=polardbx_root,host=,port=55074,schema=sbtestdb] [TDDL] Unknown command: 9, tddl version: 5.4.12-20211018

wangdong58 avatar Nov 02 '21 06:11 wangdong58