calico icon indicating copy to clipboard operation
calico copied to clipboard

calicoctl node diags try to use docker but kubernetes runs with containerd

Open Trackhe opened this issue 3 years ago • 9 comments
trafficstars

I have a problem with the connection between pods, so i find that site: https://projectcalico.docs.tigera.io/maintenance/troubleshoot/troubleshooting and tried it out, but when i use 'calicoctl node diags' i get

Collecting diagnostics
Using temp dir: /tmp/calico2622145590
Dumping netstat
Dumping routes (IPv4)
Dumping routes (IPv6)
Dumping interface info (IPv4)
Dumping interface info (IPv6)
Dumping iptables (IPv4)
Dumping iptables (IPv6)
Dumping ipsets
Failed to run command: ipset list
Error: 
Dumping ipsets (container)
Failed to run command: docker run --rm --privileged --net=host calico/node ipset list
Error: 
Copying journal for calico-node.service
Dumping felix stats
Failed to run command: pkill -SIGUSR1 felix
Error: 
Copying Calico logs
Error creating log directory: mkdir /tmp/calico2622145590/diagnostics/logs: file exists

Diags saved to /tmp/calico2622145590/diags-20220219_151350.tar.gz
If required, you can upload the diagnostics bundle to a file sharing service
such as transfer.sh using curl or similar.  For example:

    curl --upload-file /tmp/calico2622145590/diags-20220219_151350.tar.gz https://transfer.sh//tmp/calico2622145590/diags-20220219_151350.tar.gz

but i use the newest version of kubernetes and that runs with containerd. where i activated systemd to. I created the kubelet with systemd too.

Calico version:

Client Version:    v3.22.0
Git commit:        a86e41d02
Cluster Version:   v3.22.0
Cluster Type:      typha,kdd,k8s,operator,bgp,kubeadm

Kubernetes version:

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:25:17Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.4", GitCommit:"e6c093d87ea4cbb530a7b2ae91e54c0842d8308a", GitTreeState:"clean", BuildDate:"2022-02-16T12:32:02Z", GoVersion:"go1.17.7", Compiler:"gc", Platform:"linux/amd64"}

Trackhe avatar Feb 19 '22 14:02 Trackhe

This is a straight-up bug. calicoctl can no longer assume that docker is installed on kubernetes nodes.

lwr20 avatar Feb 22 '22 17:02 lwr20

I have the exact issue as described by OP - any resolution/updates?

mondragonfx avatar Sep 07 '22 11:09 mondragonfx

Still appears to be here in 3.25.0

hk135 avatar Jan 20 '23 11:01 hk135

Probably a good first issue if anyone wants to have a go. Calicoctl code is here: https://github.com/projectcalico/calico/tree/master/calicoctl

lwr20 avatar Jan 20 '23 11:01 lwr20

Can I work on this?

jodevsa avatar Mar 24 '23 19:03 jodevsa

hi the issue look not close, Can I try?

Shuimo03 avatar Sep 14 '23 07:09 Shuimo03

@jodevsa @Shuimo03 please do take a look at fixes for this!

caseydavenport avatar Sep 27 '23 21:09 caseydavenport

ok~

Shuimo03 avatar Oct 05 '23 12:10 Shuimo03

calicoctl node diags has more than one dependency or issue.

  1. Dependent Docker
  2. Dependent on netstat(net-tools), but ubuntu 22.04 have no net-tools by default, i must run apt install -y net-tools
  3. Even if there are no calico log files in the/tmp directory, an error will be reported: Error creating log directory: mkdir /tmp/calico1171669688/diagnostics/logs: file exists
  4. Failed to run command: netstat -a -n,although successful on the host, still give Failed error
  5. Failed to run command: pkill -SIGUSR1 felix,although successful on the host, still give Failed error
root@node40:~# sudo calicoctl node diags
Collecting diagnostics
Using temp dir: /tmp/calico1171669688
Dumping netstat
Failed to run command: netstat -a -n
Error: 
Dumping routes (IPv4)
Dumping routes (IPv6)
Dumping interface info (IPv4)
Dumping interface info (IPv6)
Dumping iptables (IPv4)
Dumping iptables (IPv6)
Dumping ipsets
Dumping ipsets (container)
Failed to run command: docker run --rm --privileged --net=host calico/node ipset list
Error: 
Copying journal for calico-node.service
Dumping felix stats
Failed to run command: pkill -SIGUSR1 felix
Error: 
Copying Calico logs
Error creating log directory: mkdir /tmp/calico1171669688/diagnostics/logs: file exists

Diags saved to /tmp/calico1171669688/diags-20240428_173830.tar.gz
If required, you can upload the diagnostics bundle to a file sharing service.

weironz avatar Apr 28 '24 09:04 weironz