etcd
etcd copied to clipboard
etcd-io
Robustness test kubernetes traffic sometimes doesn't issue any deletes
Bug report criteria
- [x] This bug report is not security related, security issues should be disclosed privately via etcd maintainers.
- [x] This is not a support request or question, support requests or questions should be raised in the etcd discussion forums.
- [x] You have read the etcd bug reporting guidelines.
- [x] Existing open issues along with etcd frequently asked questions have been checked and this is not a duplicate.
What happened?
Number of keys keeps growing to absurd numbers, due to no delete requests.
What did you expect to happen?
Average of keys to be kept around 10 keys, the intention of the current logic was:
- Issue only creates if number of keys drops below 5
- Issue only deletes if number of keys is over 15
Expect it might a bug with nonUniqueWriteLimiter.
How can we reproduce it (as minimally and precisely as possible)?
n/a
Anything else we need to know?
No response
Etcd version (please run commands below)
main branch
Etcd configuration (command line flags or environment variables)
TestRobustnessExploratory_Kubernetes_HighTraffic_ClusterOfSize3 test
Etcd debug information (please run commands below, feel free to obfuscate the IP address or FQDN in the output)
$ etcdctl member list -w table
# paste output here
$ etcdctl --endpoints=<member list> endpoint status -w table
# paste output here
Relevant log output
No response
I try to fix https://github.com/etcd-io/etcd/issues/17780 with more deletion. I think the root cause is that we only return token only for deletion.
https://github.com/etcd-io/etcd/blob/ce2dbbb55ea96ec02f553bd38a1d5f2a6c7a95c5/tests/robustness/traffic/etcd.go#L129-L131
When non-deletion or non-lease-revoke has more weight, they won't return the token from limiter. We should return all if we take it from limiter.
cc @henrybear327 @serathius
ping @henrybear327 if you don't have time, may I take this one? since it's related to #18099 issue. Thanks
ping @henrybear327 if you don't have time, may I take this one? since it's related to #18099 issue. Thanks
Hello @fuweid, I have unassigned myself. There was nothing specific to handover from my side.
/assign @fuweid