milvus icon indicating copy to clipboard operation
milvus copied to clipboard

[Bug]: Performance of flush, search and query degrade after etcd, querycoord pod failure chaos test in 2.1 branch

Open zhuwenxing opened this issue 2 years ago • 2 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Environment

- Milvus version:2.1.0-20220916-3c3ba55c 
- Deployment mode(standalone or cluster):
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

image

Expected Behavior

image

Steps To Reproduce

No response

Milvus Log

chaos type: pod-failure image tag: 2.1.0-20220916-3c3ba55c target pod: etcd failed job: https://qa-jenkins.milvus.io/blue/organizations/jenkins/chaos-test-for-release/detail/chaos-test-for-release/629/pipeline log:

artifacts-etcd-pod-failure-629-pytest-logs.tar.gz

artifacts-etcd-pod-failure-629-server-logs.tar.gz

chaos type: pod-failure image tag: 2.1.0-20220916-3c3ba55c target pod: querycoord failed job: https://qa-jenkins.milvus.io/blue/organizations/jenkins/chaos-test-for-release/detail/chaos-test-for-release/650/pipeline log: artifacts-querycoord-pod-failure-650-server-logs.tar.gz artifacts-querycoord-pod-failure-650-pytest-logs (1).tar.gz

Anything else?

No response

zhuwenxing avatar Sep 19 '22 08:09 zhuwenxing

/assign @czs007 /unassign

yanliang567 avatar Sep 20 '22 00:09 yanliang567

image

zhuwenxing avatar Sep 21 '22 09:09 zhuwenxing

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

stale[bot] avatar Oct 21 '22 18:10 stale[bot]