linstor-csi icon indicating copy to clipboard operation
linstor-csi copied to clipboard

Published 20 hours ago • verified publisher icon piraeusdatastore

armhf + k3s = attacher failed

Open chrstnwhlrt opened this issue 5 years ago • 0 comments

Using a raspberry pi 4 armhf k3s cluster with 3 nodes everything works fine (installed using the manual, cross compiled the images for armhf) - but the attachment of the pvs to the pods is failing.

The attacher log shows:

I1021 21:31:02.646672       1 main.go:85] Version: v2.0.0-1-g4dd6201e-dirty
I1021 21:31:02.670671       1 connection.go:151] Connecting to unix:///var/lib/csi/sockets/pluginproxy/csi.sock
I1021 21:31:05.882641       1 common.go:111] Probing CSI driver for readiness
I1021 21:31:05.882745       1 connection.go:180] GRPC call: /csi.v1.Identity/Probe
I1021 21:31:05.882793       1 connection.go:181] GRPC request: {}
I1021 21:31:05.897815       1 connection.go:183] GRPC response: {}
I1021 21:31:05.899767       1 connection.go:184] GRPC error: <nil>
I1021 21:31:05.900046       1 connection.go:180] GRPC call: /csi.v1.Identity/GetPluginInfo
I1021 21:31:05.900345       1 connection.go:181] GRPC request: {}
I1021 21:31:05.918433       1 connection.go:183] GRPC response: {"name":"linstor.csi.linbit.com","vendor_version":"v0.7.2-9-g82bdc84-dirty"}
I1021 21:31:05.919945       1 connection.go:184] GRPC error: <nil>
I1021 21:31:05.919981       1 main.go:128] CSI driver name: "linstor.csi.linbit.com"
I1021 21:31:05.920005       1 connection.go:180] GRPC call: /csi.v1.Identity/GetPluginCapabilities
I1021 21:31:05.920022       1 connection.go:181] GRPC request: {}
I1021 21:31:05.922691       1 connection.go:183] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"Service":{"type":2}}}]}
I1021 21:31:05.929761       1 connection.go:184] GRPC error: <nil>
I1021 21:31:05.929794       1 connection.go:180] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1021 21:31:05.929814       1 connection.go:181] GRPC request: {}
I1021 21:31:05.932448       1 connection.go:183] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":4}}},{"Type":{"Rpc":{"type":8}}}]}
I1021 21:31:05.947097       1 connection.go:184] GRPC error: <nil>
I1021 21:31:05.952587       1 main.go:152] CSI driver supports ControllerPublishUnpublish, using real CSI handler
I1021 21:31:05.956172       1 reflector.go:123] Starting reflector *v1.Node (10m0s) from k8s.io/client-go/informers/factory.go:133
I1021 21:31:05.956172       1 reflector.go:123] Starting reflector *v1beta1.VolumeAttachment (10m0s) from k8s.io/client-go/informers/factory.go:133
I1021 21:31:05.956494       1 controller.go:113] Starting CSI attacher
I1021 21:31:05.957211       1 reflector.go:123] Starting reflector *v1beta1.CSINode (10m0s) from k8s.io/client-go/informers/factory.go:133
I1021 21:31:05.957474       1 reflector.go:123] Starting reflector *v1.PersistentVolume (10m0s) from k8s.io/client-go/informers/factory.go:133
I1021 21:31:05.960396       1 reflector.go:161] Listing and watching *v1.PersistentVolume from k8s.io/client-go/informers/factory.go:133
I1021 21:31:05.960398       1 reflector.go:161] Listing and watching *v1beta1.VolumeAttachment from k8s.io/client-go/informers/factory.go:133
I1021 21:31:05.960397       1 reflector.go:161] Listing and watching *v1.Node from k8s.io/client-go/informers/factory.go:133
I1021 21:31:05.960499       1 reflector.go:161] Listing and watching *v1beta1.CSINode from k8s.io/client-go/informers/factory.go:133
I1021 21:31:06.056800       1 shared_informer.go:123] caches populated
I1021 21:31:06.057039       1 controller.go:205] Started PV processing "pvc-51ab78fc-392b-4c43-b203-93b40d166019"
I1021 21:31:06.057212       1 csi_handler.go:444] CSIHandler: processing PV "pvc-51ab78fc-392b-4c43-b203-93b40d166019"
I1021 21:31:06.057366       1 csi_handler.go:448] CSIHandler: processing PV "pvc-51ab78fc-392b-4c43-b203-93b40d166019": no deletion timestamp, ignoring
I1021 21:31:06.057243       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:06.057658       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:06.057769       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:06.057911       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:06.058125       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:31:06.058305       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:06.057573       1 controller.go:205] Started PV processing "pvc-e1957a70-cecc-403f-aba7-e66a2faf7d7c"
I1021 21:31:06.058656       1 csi_handler.go:444] CSIHandler: processing PV "pvc-e1957a70-cecc-403f-aba7-e66a2faf7d7c"
I1021 21:31:06.058998       1 csi_handler.go:480] CSIHandler: processing PV "pvc-e1957a70-cecc-403f-aba7-e66a2faf7d7c": VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" found
I1021 21:31:06.103007       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:06.103113       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:31:06.104358       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:31:07.103423       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:07.103492       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:07.103526       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:07.103657       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:07.103701       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:31:07.103731       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:07.116285       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:07.116342       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:31:07.118022       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:31:09.116676       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:09.116779       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:09.116853       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:09.117084       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:09.117177       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:31:09.117241       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:09.132317       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:09.132424       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:31:09.136760       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:31:13.133045       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:13.133235       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:13.133343       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:13.133747       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:13.133923       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:31:13.134059       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:13.153233       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:13.153356       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:31:13.155010       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:31:21.153871       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:21.154094       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:21.154211       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:21.154742       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:21.154926       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:31:21.155037       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:21.174187       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:31:21.175257       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:21.175353       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:31:37.175745       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:37.175854       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:37.175924       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:37.176171       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:37.176284       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:31:37.176351       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:37.192899       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:31:37.193014       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:31:37.195716       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:32:09.193532       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:32:09.193656       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:32:09.194171       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:32:09.195109       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:32:09.195334       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:32:09.195500       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:32:09.211456       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:32:09.211572       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:32:09.215098       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:33:13.212001       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:33:13.212203       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:33:13.212264       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:33:13.212441       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:33:13.212542       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:33:13.212611       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:33:13.230631       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:33:13.230745       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:33:13.233589       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:35:21.231335       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:35:21.231540       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:35:21.231610       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:35:21.231766       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:35:21.231860       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:35:21.231924       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:35:21.249026       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:35:21.249137       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:35:21.252320       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:38:17.025112       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1beta1.VolumeAttachment total 9 items received
I1021 21:39:07.008475       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1beta1.CSINode total 0 items received
I1021 21:39:25.025166       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1.PersistentVolume total 0 items received
I1021 21:39:37.249616       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:39:37.249816       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:39:37.249884       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:39:37.250132       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:39:37.250238       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:39:37.250307       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:39:37.275578       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:39:37.275679       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:39:37.294831       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:40:48.020597       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1.Node total 27 items received
I1021 21:41:06.022835       1 reflector.go:235] k8s.io/client-go/informers/factory.go:133: forcing resync
I1021 21:41:06.022988       1 controller.go:205] Started PV processing "pvc-51ab78fc-392b-4c43-b203-93b40d166019"
I1021 21:41:06.023028       1 csi_handler.go:444] CSIHandler: processing PV "pvc-51ab78fc-392b-4c43-b203-93b40d166019"
I1021 21:41:06.023111       1 csi_handler.go:448] CSIHandler: processing PV "pvc-51ab78fc-392b-4c43-b203-93b40d166019": no deletion timestamp, ignoring
I1021 21:41:06.023171       1 controller.go:205] Started PV processing "pvc-e1957a70-cecc-403f-aba7-e66a2faf7d7c"
I1021 21:41:06.023201       1 csi_handler.go:444] CSIHandler: processing PV "pvc-e1957a70-cecc-403f-aba7-e66a2faf7d7c"
I1021 21:41:06.023265       1 reflector.go:235] k8s.io/client-go/informers/factory.go:133: forcing resync
I1021 21:41:06.023350       1 csi_handler.go:480] CSIHandler: processing PV "pvc-e1957a70-cecc-403f-aba7-e66a2faf7d7c": VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" found
I1021 21:41:06.023366       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:41:06.023400       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:41:06.023425       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:41:06.023478       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:41:06.023517       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:41:06.023544       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:41:06.030694       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:41:06.030746       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:41:06.032592       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:44:37.276091       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:44:37.276239       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:44:37.276302       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:44:37.276537       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:44:37.276631       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:44:37.276696       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:44:37.300483       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:44:37.300602       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:44:37.300767       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:44:44.029151       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1.PersistentVolume total 0 items received
I1021 21:45:24.027418       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1beta1.VolumeAttachment total 3 items received
I1021 21:46:34.024944       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1.Node total 18 items received
I1021 21:47:33.012910       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1beta1.CSINode total 0 items received
I1021 21:49:37.300979       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:49:37.301329       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:49:37.301554       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:49:37.302002       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:49:37.302262       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:49:37.302448       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:49:37.319575       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:49:37.319890       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:49:37.323078       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:50:13.033202       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1.PersistentVolume total 0 items received
I1021 21:51:06.023140       1 reflector.go:235] k8s.io/client-go/informers/factory.go:133: forcing resync
I1021 21:51:06.023424       1 controller.go:205] Started PV processing "pvc-51ab78fc-392b-4c43-b203-93b40d166019"
I1021 21:51:06.023524       1 csi_handler.go:444] CSIHandler: processing PV "pvc-51ab78fc-392b-4c43-b203-93b40d166019"
I1021 21:51:06.023677       1 csi_handler.go:448] CSIHandler: processing PV "pvc-51ab78fc-392b-4c43-b203-93b40d166019": no deletion timestamp, ignoring
I1021 21:51:06.023787       1 controller.go:205] Started PV processing "pvc-e1957a70-cecc-403f-aba7-e66a2faf7d7c"
I1021 21:51:06.023845       1 csi_handler.go:444] CSIHandler: processing PV "pvc-e1957a70-cecc-403f-aba7-e66a2faf7d7c"
I1021 21:51:06.024220       1 reflector.go:235] k8s.io/client-go/informers/factory.go:133: forcing resync
I1021 21:51:06.024334       1 csi_handler.go:480] CSIHandler: processing PV "pvc-e1957a70-cecc-403f-aba7-e66a2faf7d7c": VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" found
I1021 21:51:06.025001       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:51:06.025234       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:51:06.025446       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:51:06.025816       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:51:06.026058       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:51:06.026280       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:51:06.041269       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:51:06.041375       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:51:06.046043       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:51:54.030065       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1beta1.VolumeAttachment total 2 items received
I1021 21:54:08.032449       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1.Node total 24 items received
I1021 21:54:37.320513       1 controller.go:175] Started VA processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:54:37.320616       1 csi_handler.go:89] CSIHandler: processing VA "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:54:37.320691       1 csi_handler.go:140] Starting detach operation for "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:54:37.320914       1 csi_handler.go:147] Detaching "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:54:37.321007       1 csi_handler.go:550] Can't get CSINode host2: csinode.storage.k8s.io "host2" not found
I1021 21:54:37.321074       1 csi_handler.go:428] Saving detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:54:37.339042       1 csi_handler.go:439] Saved detach error to "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430"
I1021 21:54:37.339204       1 csi_handler.go:99] Error processing "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430": failed to detach: node "host2" has no NodeID annotation
I1021 21:54:37.339290       1 controller.go:141] Ignoring VolumeAttachment "csi-ab1e8c5f78cbcad5b31d7bc02b5a0b2f342395a46b4e60b1f1a232c4e6068430" change
I1021 21:56:17.040482       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1.PersistentVolume total 0 items received
I1021 21:56:37.019863       1 reflector.go:370] k8s.io/client-go/informers/factory.go:133: Watch close - *v1beta1.CSINode total 0 items received

I supposed the CSINodeInfo feature is no working correctly in k3s, is this correct or does the log point to another problem?

Link the the (possible) k3s bug

chrstnwhlrt avatar Oct 22 '19 08:10 chrstnwhlrt