etcdfs icon indicating copy to clipboard operation
etcdfs copied to clipboard

When used with SSL etcdfs throws error after a while

Open DicksengA opened this issue 4 years ago • 6 comments

Able to mount with etcd client auth enabled. After approximately 3 mins, unable to interact with etcd. Error displays Failed to list keys from etcd, error="context deadline exceeded"

DicksengA avatar Dec 08 '20 08:12 DicksengA

This means a timeout error when reading from etcd. Could you try a larger timeout by specifying --read-timeout? maybe the default one (3s) is not enough in you case.

polyrabbit avatar Dec 08 '20 08:12 polyrabbit

@polyrabbit Tried with 60s. Still getting the error. Weird thing is i works for a short duration of time before starting to display context deadline exceeded

DicksengA avatar Dec 09 '20 01:12 DicksengA

Could you add another parameter -v to get a more verbose log, and paste the whole log here?

polyrabbit avatar Dec 09 '20 03:12 polyrabbit

INFO[04:39:54] Mounted to "/home/ec2-user/etcdfs", use ctrl+c to terminate. DEBU[04:40:02] Node Readdir path=/ DEBU[04:40:02] Node Lookup path=/ansible-playbook DEBU[04:40:02] Node Lookup path=/aws DEBU[04:40:02] Node Lookup path=/etcd-openshift DEBU[04:40:21] Node Readdir path=/ DEBU[04:40:21] Node Lookup path=/ansible-playbook DEBU[04:40:21] Node Lookup path=/aws DEBU[04:40:21] Node Lookup path=/etcd-openshift DEBU[04:40:24] Node Readdir path=/ DEBU[04:40:24] Node Lookup path=/ansible-playbook DEBU[04:40:24] Node Lookup path=/aws DEBU[04:40:24] Node Lookup path=/etcd-openshift DEBU[04:40:24] Node Readdir path=/ DEBU[04:40:24] Node Lookup path=/ansible-playbook DEBU[04:40:24] Node Lookup path=/aws DEBU[04:40:24] Node Lookup path=/etcd-openshift DEBU[04:40:24] Node Readdir path=/ DEBU[04:40:24] Node Lookup path=/ansible-playbook DEBU[04:40:24] Node Lookup path=/aws DEBU[04:40:24] Node Lookup path=/etcd-openshift DEBU[04:40:25] Node Lookup path=/ansible-playbook DEBU[04:40:26] Node Readdir path=/ansible-playbook/ DEBU[04:40:26] Node Lookup path=/ansible-playbook/etcd-server.yml DEBU[04:40:26] Node Lookup path=/ansible-playbook/setup-statemanager.yml DEBU[04:40:26] Node Lookup path=/ansible-playbook/ec2-baseline.yml DEBU[04:40:39] Node Lookup path=/ansible-playbook DEBU[04:40:40] Node Readdir path=/ DEBU[04:40:40] Node Lookup path=/ansible-playbook DEBU[04:40:40] Node Lookup path=/aws DEBU[04:40:40] Node Lookup path=/etcd-openshift DEBU[04:41:08] Node Readdir path=/ ERRO[04:41:18] Failed to list keys from etcd error="context deadline exceeded" path=/ DEBU[04:41:22] Node Readdir path=/ ERRO[04:41:32] Failed to list keys from etcd error="context deadline exceeded" path=/

DicksengA avatar Dec 09 '20 04:12 DicksengA

I have the same problem and I don't use SSL. Finally, I found out that the auto-sync of etcd caused it and if I commented the line AutoSyncInterval: time.Minute,, then everything works fine. I run etcd through docker, I‘m not sure if there is something wrong with my config.

xwjdsh avatar Jul 16 '21 12:07 xwjdsh

My etcd runs on a remote VM, and I don't have such an issue. How many entries do you have under that etcd prefix? One possible way to diagnose this issue is to open debug port and see what goroutines are doing via /debug/pprof/goroutine. Are there any pending connections that consuming too much resources?

polyrabbit avatar Jul 17 '21 10:07 polyrabbit