aws-fsx-csi-driver icon indicating copy to clipboard operation
aws-fsx-csi-driver copied to clipboard

Allow configurable timeout for WaitForFileSystemAvailable

Open jon-rei opened this issue 7 months ago • 21 comments

Is your feature request related to a problem? Please describe.

We are starting to use Scratch 2 FSx FileSystems and are doing so successfully for small ones. But when we try to create FSx volumes >50TB we always run into timeouts:

E0701 17:57:05.334673       1 driver.go:104] "GRPC error" err="rpc error: code = Aborted desc = Create volume request for pvc-1234 is already in progress"
E0701 17:57:06.262188       1 driver.go:104] "GRPC error" err=<
	rpc error: code = Internal desc = Filesystem is not ready: RequestCanceled: request context canceled
	caused by: context canceled
 >

It seems that the timeout is always 5 minutes, but creating a 72TB volume for example takes ~12 minutes.

Describe the solution you'd like in detail

It would be great if this timeout could be increased or made configurable since it's hardcoded here.

Describe alternatives you've considered

Right now we are trying to create the filesystems manually and then create a PV + PVC. But we could really create the volumes dynamically for the pipelines we want to run.

jon-rei avatar Jul 01 '24 18:07 jon-rei