csi sanity issues

[hemalathagajendran]

Describe the bug

To fix csi sanity issues which requires analysis and considerable effort

How to Reproduce?

Run csi-sanity command in real time cluster where CSI driver runs

1. [FAIL] Controller Service [Controller Server] ValidateVolumeCapabilities [It] should fail when the requested volume does not exist
./csi-sanity  --ginkgo.focus-file=controller.go:825-842 --csi.endpoint=/var/lib/kubelet/plugins/spectrumscale.csi.ibm.com/csi.sock  -csi.testvolumeparameters parameter.yaml  -csi.createmountpathcmd ./mkdir_in_pod.sh -csi.removemountpathcmd ./rmdir_in_pod.sh

2.  [FAIL] Controller Service [Controller Server] ControllerPublishVolume [It] should fail when the node does not exist
  /root/csi-test/pkg/sanity/controller.go:998


I1121 09:49:24.489436       1 utils.go:58] [b4d6f517-3426-4f73-914c-3e8bec098e40] GRPC request: volume_id:"fake-vol-id-2f293426-7" volume_capabilities:<mount:<> access_mode:<mode:SINGLE_NODE_WRITER > >
I1121 09:49:24.489530       1 controllerserver.go:1614] [b4d6f517-3426-4f73-914c-3e8bec098e40] ValidateVolumeCapabilities called with req: &csi.ValidateVolumeCapabilitiesRequest{VolumeId:"fake-vol-id-2f293426-7", VolumeContext:map[string]string(nil), VolumeCapabilities:[]*csi.VolumeCapability{(*csi.VolumeCapability)(0xc0003b8a80)}, Parameters:map[string]string(nil), Secrets:map[string]string(nil), XXX_NoUnkeyedLiteral:struct {}{}, XXX_unrecognized:[]uint8(nil), XXX_sizecache:0}
I1121 09:49:24.489557       1 utils.go:63] [b4d6f517-3426-4f73-914c-3e8bec098e40] GRPC response:

Expected behavior for 1st issue:

"Not Found" error is expected.

[FAIL] Node Service NodeGetVolumeStats [It] should fail when volume does not exist on the specified path /root/csi-test/pkg/sanity/node.go:282 [FAIL] Node Service [It] should work /root/csi-test/pkg/sanity/node.go:163 [FAIL] Node Service [It] should be idempotent /root/csi-test/pkg/sanity/node.go:163