pd icon indicating copy to clipboard operation
pd copied to clipboard
verified publisher icon tikv

TestConcurrentlyReset is unstable

Open lhy1024 opened this issue 2 years ago • 4 comments

Flaky Test

Which jobs are failing

2023-08-04T10:00:27.2580986Z === RUN   TestMicroserviceTSOServer/TestConcurrentlyReset
2023-08-04T10:00:27.2583758Z [2023/08/04 10:00:27.258 +00:00] [INFO] [handler.go:37] [reset-ts] [new-ts=443323050166321152] [ignore-smaller=false] [skip-upper-bound-check=false] [keyspace-group-id=0]
2023-08-04T10:00:27.2653657Z [2023/08/04 10:00:27.265 +00:00] [INFO] [handler.go:37] [reset-ts] [new-ts=443323050166321152] [ignore-smaller=false] [skip-upper-bound-check=false] [keyspace-group-id=0]
2023-08-04T10:00:27.2657681Z     server_test.go:115: 
2023-08-04T10:00:27.2658891Z         	Error Trace:	/home/runner/work/pd/pd/tests/integrations/tso/server_test.go:115
2023-08-04T10:00:27.2660538Z         	            				/home/runner/work/pd/pd/tests/integrations/tso/server_test.go:161
2023-08-04T10:00:27.2661801Z         	            				/opt/hostedtoolcache/go/1.20.1/x64/src/runtime/asm_amd64.s:1598
2023-08-04T10:00:27.2663310Z         	Error:      	Error "[PD:tso:ErrResetUserTimestamp]reset user timestamp failed, the specified ts is too larger than now" does not contain "is smaller than now"
2023-08-04T10:00:27.2664548Z         	Test:       	TestMicroserviceTSOServer/TestConcurrentlyReset
2023-08-04T10:00:27.2665513Z     server_test.go:115: 
2023-08-04T10:00:27.2666398Z         	Error Trace:	/home/runner/work/pd/pd/tests/integrations/tso/server_test.go:115
2023-08-04T10:00:27.2667977Z         	            				/home/runner/work/pd/pd/tests/integrations/tso/server_test.go:161
2023-08-04T10:00:27.2669182Z         	            				/opt/hostedtoolcache/go/1.20.1/x64/src/runtime/asm_amd64.s:1598
2023-08-04T10:00:27.2670665Z         	Error:      	Error "[PD:tso:ErrResetUserTimestamp]reset user timestamp failed, the specified ts is too larger than now" does not contain "is smaller than now"
2023-08-04T10:00:27.2671654Z         	Test:       	TestMicroserviceTSOServer/TestConcurrentlyReset

CI link

https://github.com/tikv/pd/actions/runs/5760861853/job/15617635075?pr=6888

Reason for failure (if possible)

Anything else

lhy1024 avatar Aug 06 '23 16:08 lhy1024

It is the same error with https://github.com/tikv/pd/issues/6385

lhy1024 avatar Aug 29 '23 04:08 lhy1024

https://github.com/tikv/pd/actions/runs/6258743955/job/16993367750?pr=7120

lhy1024 avatar Sep 21 '23 08:09 lhy1024

https://github.com/tikv/pd/actions/runs/6663065233/job/18108396355?pr=7272

lhy1024 avatar Oct 27 '23 05:10 lhy1024

meet again

2024-09-10T07:48:20.9874895Z === RUN   TestMicroserviceTSOServer/TestConcurrentlyReset
2024-09-10T07:48:20.9876116Z [2024/09/10 07:48:08.004 +00:00] [INFO] [server.go:326] [reset-ts] [new-ts=452448613303320576] [ignore-smaller=false] [skip-upper-bound-check=false] [keyspace-group-id=0]
2024-09-10T07:48:20.9877664Z [2024/09/10 07:48:08.004 +00:00] [INFO] [server.go:326] [reset-ts] [new-ts=452448613303320576] [ignore-smaller=false] [skip-upper-bound-check=false] [keyspace-group-id=0]
2024-09-10T07:48:20.9878583Z     server_test.go:116: 
2024-09-10T07:48:20.9879451Z         	Error Trace:	/home/runner/work/pd/pd/tests/integrations/tso/server_test.go:116
2024-09-10T07:48:20.9880666Z         	            				/home/runner/work/pd/pd/tests/integrations/tso/server_test.go:162
2024-09-10T07:48:20.9881740Z         	            				/opt/hostedtoolcache/go/1.21.13/x64/src/runtime/asm_amd64.s:1650
2024-09-10T07:48:20.9883167Z         	Error:      	Error "[PD:tso:ErrResetUserTimestamp]reset user timestamp failed, the specified ts is too larger than now" does not contain "is smaller than now"
2024-09-10T07:48:20.9884364Z         	Test:       	TestMicroserviceTSOServer/TestConcurrentlyReset
2024-09-10T07:48:20.9884896Z     server_test.go:116: 
2024-09-10T07:48:20.9885725Z         	Error Trace:	/home/runner/work/pd/pd/tests/integrations/tso/server_test.go:116
2024-09-10T07:48:20.9886907Z         	            				/home/runner/work/pd/pd/tests/integrations/tso/server_test.go:162
2024-09-10T07:48:20.9888077Z         	            				/opt/hostedtoolcache/go/1.21.13/x64/src/runtime/asm_amd64.s:1650
2024-09-10T07:48:20.9889539Z         	Error:      	Error "[PD:tso:ErrResetUserTimestamp]reset user timestamp failed, the specified ts is too larger than now" does not contain "is smaller than now"
2024-09-10T07:48:20.9890672Z         	Test:       	TestMicroserviceTSOServer/TestConcurrentlyReset

https://github.com/tikv/pd/actions/runs/10788038239/attempts/1?pr=8613

HuSharp avatar Sep 10 '24 07:09 HuSharp

https://github.com/tikv/pd/actions/runs/11362521961/job/31604561100?pr=8707

okJiang avatar Oct 16 '24 09:10 okJiang