[Bug]: lots of error “no such table sbtest.sbtestX” in sysbench mixed test during stability test on standlone mode

[aressu1985]

Is there an existing issue for the same bug?

• [X] I have checked the existing issues.

Branch Name

1.1-dev

Commit ID

ba86e9ca56e232305fc07cfaf4727d8480ed54aa

Other Environment Information

- Hardware parameters:16C 64G
- OS type:
- Others:

Actual Behavior

During stability test on standlone, there lots of the following error in sysbench mixed cases ErrorMessage : no such table sbtest.sbtest5 ErrorMessage : no such table sbtest.sbtest8 ErrorMessage : no such table sbtest.sbtest8 ErrorMessage : no such table sbtest.sbtest4

log is more than 1G, if wanted, contanct me offline,the part is : {"level":"ERROR","time":"2024/02/25 08:23:22.096200 +0800","name":"log-service.frontend","caller":"frontend/mysql_cmd_executor.go:3071","msg":"waiter is paused","uuid":"7c4dccb4-4d3c-41f8-b482-5251dc7a41bf","session_info":"connectionId 219|127.0.0.1:57118|{account sys:dump:moadmin -- 0:1:0}|goRoutineId 7887|018dd936-a548-7b0b-9a93-f5854b17f932","session_id":"018dd936-a548-7b0b-9a93-f5854b17f932","statement_id":"018ddda5-1cf0-7d59-b0c0-ef588c24e485"} {"level":"ERROR","time":"2024/02/25 08:23:22.096225 +0800","name":"log-service.frontend","caller":"frontend/util.go:511","msg":"query trace status","uuid":"7c4dccb4-4d3c-41f8-b482-5251dc7a41bf","connection_id":219,"statement":"UPDATE sbtest6 SET k=k+100 WHERE id = 19049","status":"fail","error":"waiter is paused","span":{"trace_id":"a3073695-ca52-f7a7-bb10-25aca4f33af7","span_id":"95299fc99f010c2b"},"session_info":"connectionId 219|127.0.0.1:57118|{account sys:dump:moadmin -- 0:1:0}|goRoutineId 7887|018dd936-a548-7b0b-9a93-f5854b17f932","session_id":"018dd936-a548-7b0b-9a93-f5854b17f932","statement_id":"018ddda5-1cf0-7d59-b0c0-ef588c24e485"} {"level":"ERROR","time":"2024/02/25 08:23:22.096232 +0800","caller":"client/timestamp_waiter.go:137","msg":"error: waiter is paused"} {"level":"ERROR","time":"2024/02/25 08:23:22.096258 +0800","name":"log-service.frontend","caller":"frontend/txn.go:442","msg":"Failed to get transaction","uuid":"7c4dccb4-4d3c-41f8-b482-5251dc7a41bf","error":"waiter is paused","session_info":"connectionId 214|127.0.0.1:57108|{account sys:dump:moadmin -- 0:1:0}|goRoutineId 7915|018dd936-a4cf-7322-918a-497985a20082","session_id":"018dd936-a4cf-7322-918a-497985a20082","statement_id":"018ddda5-1cf0-7af3-9cb9-8d6a9726b471"} {"level":"ERROR","time":"2024/02/25 08:23:22.096277 +0800","name":"log-service.frontend","caller":"frontend/mysql_cmd_executor.go:3071","msg":"waiter is paused","uuid":"7c4dccb4-4d3c-41f8-b482-5251dc7a41bf","session_info":"connectionId 214|127.0.0.1:57108|{account sys:dump:moadmin -- 0:1:0}|goRoutineId 7915|018dd936-a4cf-7322-918a-497985a20082","session_id":"018dd936-a4cf-7322-918a-497985a20082","statement_id":"018ddda5-1cf0-7af3-9cb9-8d6a9726b471"} {"level":"ERROR","time":"2024/02/25 08:23:22.096301 +0800","name":"log-service.frontend","caller":"frontend/util.go:511","msg":"query trace status","uuid":"7c4dccb4-4d3c-41f8-b482-5251dc7a41bf","connection_id":214,"statement":"UPDATE sbtest3 SET k=k+100 WHERE id = 40833","status":"fail","error":"waiter is paused","span":{"trace_id":"bae0c5a3-1a85-9375-08ab-b49e4fd642c7","span_id":"655ea40821d5b122"},"session_info":"connectionId 214|127.0.0.1:57108|{account sys:dump:moadmin -- 0:1:0}|goRoutineId 7915|018dd936-a4cf-7322-918a-497985a20082","session_id":"018dd936-a4cf-7322-918a-497985a20082","statement_id":"018ddda5-1cf0-7af3-9cb9-8d6a9726b471"} {"level":"INFO","time":"2024/02/25 08:23:22.096566 +0800","caller":"disttae/logtail_consumer.go:531","msg":"[logtail-consumer] 1708746016182506732dd1dccb4-4d3c-41f8-b482-5251dc7a41bf: abort all running transactions finished"} {"level":"ERROR","time":"2024/02/25 08:23:22.099933 +0800","caller":"disttae/engine.go:205","msg":"error: the transaction 018ddda51c9e7b958a8d8213268024c2 has been committed or aborted"} {"level":"ERROR","time":"2024/02/25 08:23:22.099985 +0800","caller":"frontend/authenticate.go:3261","msg":"Get Subscription database sbtest meta error: the transaction 018ddda51c9e7b958a8d8213268024c2 has been committed or aborted"} {"level":"ERROR","time":"2024/02/25 08:23:22.100023 +0800","caller":"frontend/authenticate.go:3262","msg":"error: not connect to a database","span":{"trace_id":"884a8f66-a66d-0a05-3ea4-9956c468be88","span_id":"888ccf60c28efd5d"}} {"level":"ERROR","time":"2024/02/25 08:23:22.100089 +0800","caller":"plan/build_constraint_util.go:250","msg":"error: no such table sbtest.sbtest4","span":{"trace_id":"884a8f66-a66d-0a05-3ea4-9956c468be88","span_id":"888ccf60c28efd5d"}} {"level":"ERROR","time":"2024/02/25 08:23:22.100129 +0800","name":"log-service.frontend","caller":"frontend/mysql_cmd_executor.go:3071","msg":"no such table sbtest.sbtest4","uuid":"7c4dccb4-4d3c-41f8-b482-5251dc7a41bf","session_info":"connectionId 196|127.0.0.1:57068|{account sys:dump:moadmin -- 0:1:0}|goRoutineId 7988|018dd936-a2cd-71e5-bab5-41a66a7d04eb","session_id":"018dd936-a2cd-71e5-bab5-41a66a7d04eb","statement_id":"018ddda5-1cf3-735c-a0c7-2c81ff6f4261","txn_id":"018ddda51c9e7b958a8d8213268024c2/Aborted/S:1708820600129552911-1"} {"level":"ERROR","time":"2024/02/25 08:23:22.100186 +0800","name":"log-service.frontend","caller":"frontend/util.go:511","msg":"query trace status","uuid":"7c4dccb4-4d3c-41f8-b482-5251dc7a41bf","connection_id":196,"statement":"delete from sbtest4 where id = 66689584","status":"fail","error":"no such table sbtest.sbtest4","span":{"trace_id":"884a8f66-a66d-0a05-3ea4-9956c468be88","span_id":"888ccf60c28efd5d"},"session_info":"connectionId 196|127.0.0.1:57068|{account sys:dump:moadmin -- 0:1:0}|goRoutineId 7988|018dd936-a2cd-71e5-bab5-41a66a7d04eb","session_id":"018dd936-a2cd-71e5-bab5-41a66a7d04eb","statement_id":"018ddda5-1cf3-735c-a0c7-2c81ff6f4261"} {"level":"ERROR","time":"2024/02/25 08:23:22.100323 +0800","caller":"client/operator.go:600","msg":"error: the transaction 018ddda51ca170fcb182ebb4277e9aff has been committed or aborted"} {"level":"ERROR","time":"2024/02/25 08:23:22.100382 +0800","name":"log-service.frontend","caller":"frontend/txn.go:353","msg":"CommitTxn: txn operator commit failed","uuid":"7c4dccb4-4d3c-41f8-b482-5251dc7a41bf","txnId":"018ddda51ca170fcb182ebb4277e9aff/Aborted/S:1708820600129552911-1","error":"the transaction 018ddda51ca170fcb182ebb4277e9aff has been committed or aborted","session_info":"connectionId 157|127.0.0.1:56984|{account sys:dump:moadmin -- 0:1:0}|goRoutineId 7648|018dd936-9dbc-74a5-b0ee-33ad289bbfaf","session_id":"018dd936-9dbc-74a5-b0ee-33ad289bbfaf","statement_id":"018ddda5-1ca1-746d-8ef8-759854170411"}

Expected Behavior

No response

Steps to Reproduce

run stability test on standlone mode
1. tpch 10G loop queries
2. sysbench mixed cases with 30 threads
   insert and delte
   point select
   update by pk
3. tpcc 10-10 terminals

Additional information

No response

[jensenojs]

本地试了一下, 没复现成功

[Ariznawlll]

@nnsgmsone 辛苦帮忙看一下呀.

[nnsgmsone]

处理数据正确性问题中

[nnsgmsone]

no process

[nnsgmsone]

no process

[nnsgmsone]

处理事务泄露中

[nnsgmsone]

no process

[nnsgmsone]

no process

[sukki37]

Recently, the stability tests have not reproduced the issue; let's close it for now.