matrixone
matrixone copied to clipboard
matrixorigin
[Bug]: merge run failed frequently caused by 'connection closed'
Is there an existing issue for the same bug?
- [X] I have checked the existing issues.
Branch Name
main
Commit ID
dea366b31f593cdc7fcf890feba0b898facf0554
Other Environment Information
- Hardware parameters:
- OS type:
- Others:
Actual Behavior
https://github.com/matrixorigin/matrixone/actions/runs/9007728379
Expected Behavior
No response
Steps to Reproduce
mergerun test
Additional information
No response
#15777 may cause a huge increase in in-use objects of frontend, which in turn will lead to an increase in CPU consumption by GC marking, causing the MO lockservice to be unable to obtain CPU in time, ultimately resulting in a timeout.
https://grafana.ci.matrixorigin.cn/explore?panes=%7B%22M_o%22:%7B%22datasource%22:%22pyroscope%22,%22queries%22:%5B%7B%22groupBy%22:%5B%5D,%22labelSelector%22:%22%7Bnamespace%3D%5C%22mo-checkin-regression-15777%5C%22%7D%22,%22queryType%22:%22both%22,%22refId%22:%22A%22,%22profileTypeId%22:%22memory:inuse_objects:count:space:bytes%22,%22datasource%22:%7B%22type%22:%22grafana-pyroscope-datasource%22,%22uid%22:%22pyroscope%22%7D%7D%5D,%22range%22:%7B%22from%22:%221715201340000%22,%22to%22:%221715201400000%22%7D%7D,%22I-K%22:%7B%22datasource%22:%22pyroscope%22,%22queries%22:%5B%7B%22groupBy%22:%5B%5D,%22labelSelector%22:%22%7Bnamespace%3D%5C%22mo-checkin-regression-15910%5C%22%7D%22,%22queryType%22:%22both%22,%22refId%22:%22A%22,%22profileTypeId%22:%22memory:inuse_objects:count:space:bytes%22,%22datasource%22:%7B%22type%22:%22grafana-pyroscope-datasource%22,%22uid%22:%22pyroscope%22%7D%7D%5D,%22range%22:%7B%22from%22:%221715198880000%22,%22to%22:%221715198940000%22%7D%7D%7D&schemaVersion=1&orgId=1
#https://github.com/matrixorigin/matrixone/pull/16551 have solved this problem
