feature request: supporting filestore

[atraydixit]

related to the thread on creating mounting disk support with respect to support nfs filestore.

seems like I bump into a an issue with the following error: statd: unrecognized service mount.nfs: rpc.statd is not running but is required for remote locking. mount.nfs: Either use '-o nolock' to keep locks local, or start statd.

[wnojopra]

Hi @atraydixit ,

I'm not familiar with the thread you mentioned. Can you point me to it? I don't have context on what you did to run into those errors.

In general I'm interested in supporting nfs filestore. But it may be something we have to request the Pipelines API to support.

[atraydixit]

just the thread re: mounting disks https://github.com/DataBiosphere/dsub/issues/132

in the case of filestore it should be feasible to perform the mount operation within a script being run on a client VM created by dsub

[wnojopra]

Thanks @atraydixit for clarity and for the feature request. We'll need to add support to mounting Filestore fileshares, similar to how we mount a persistent disk. I don't see any documentation on this in the Pipelines API documentation, so we may have to request support from there.

[atraydixit]

got it thanks!

somewhat relatedly, I'm having some issues using the --ssh flag to enable interfacing with the dsub generated instances. has this feature been full developed.

Permission denied (publickey). ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255].

any ideas.

additionally, is their build in functionality for using the --no-address flag to avoid excessive external IP address requirements?

[mbookman]

Hi @atraydixit !

On the original request about Filestore support, I wanted to ask - is this actually a feature that you'd be likely to use? I ask because Cloud Filestore ($0.20 /GB /mo) is at least 10 times as expensive as using a Regional Cloud Storage bucket ($0.02 / GB /mo). I say "at least" because with Cloud Filestore, you pay for provisioned space whether you use it or not, while with GCS you only pay for what you use.

With regards to --ssh, did you resolve your issue? It is a feature used regularly. Are you able to ssh into other VMs in the same cloud project?

Lastly on --no-address flag, dsub does have a flag --use-private-address which can be used if you have configured your project networking appropriately.

Thanks.

[atraydixit]

thanks for the reply and sorry for the delay on my end.

re: filestore - mostly wanted to use it in the context of a single large file that each instance needed read access to without having to have each one have the storage for the file. ex: 50 GB file where each of 100 VMs is trying to read from it means I'd need >5TB of storage allocated.

w.r.t --ssh yes I've done that before just not with one launched through dsub.

re: --no-address the --use-private-address flag was useful.

thanks!