Matt Dainty
Matt Dainty
This is looking suspiciously like the same problem I'm hitting. What does `kubectl logs tectonic-console-6d4578d847-ff664 --namespace=tectonic-system` show for you? I'm getting the following for the same three crashing pods: ```...
As far as I can tell the Kubernetes cluster itself is working, I can use `kubectl` to inspect the cluster and I find all of the nodes and pods, etc....
@squat Sure, that's how I would expect it be running, however on my workers there's nothing listening on port 443 but I found it is listening on port 32000. For...
Is there some debugging or commands I can run to help get you more info? Edit: I found I still had this output in a terminal: ``` $ kubectl describe...
Thank you @squat for the explanation, I reverted #2911 and rebuilt the cluster and now the console works as expected. Is it worth me raising this as a separate issue?
I've raised #3080.
Hi @ktock Here's the output from `du -hxd 2 /var/lib/containerd-stargz-grpc/`: ``` [root@ip-10-202-107-137 ~]# du -hxd 2 /var/lib/containerd-stargz-grpc/ 34G /var/lib/containerd-stargz-grpc/snapshotter/snapshots 34G /var/lib/containerd-stargz-grpc/snapshotter 0 /var/lib/containerd-stargz-grpc/stargz/httpcache 0 /var/lib/containerd-stargz-grpc/stargz/fscache 0 /var/lib/containerd-stargz-grpc/stargz 34G /var/lib/containerd-stargz-grpc/ ```...
> @bodgit Thanks for the info. > > > 34G /var/lib/containerd-stargz-grpc/snapshotter/snapshots > > What does consume the large space under this directory? Are there many snapshot dirs or is there...
I think I've found the problem. I noticed we were getting this message logged often: ``` kubelet: E0820 03:07:11.954394 3800 cri_stats_provider.go:455] "Failed to get the info of the filesystem with...