dstack icon indicating copy to clipboard operation
dstack copied to clipboard

[Feature]: Allow `privileged` mode for docker on GCP

Open rejuvyesh opened this issue 1 year ago • 1 comments

Problem

GCSFuse allows mounting a remote GCS bucket as a local filesystem and is a standard thing to do in HPC contexts. https://github.com/GoogleCloudPlatform/gcsfuse/issues/566 means that mounting a fuse system inside docker requires running it in privileged mode. It seems current instance configuration chooses to not allow passing in any config to docker run.

Solution

For TPU support, dstack already enables privileged mode for the docker image. We just need to allow that to happen for standard GPUs on GCP as well.

Workaround

No response

Would you like to help us implement this feature by sending a PR?

Yes

rejuvyesh avatar Jul 07 '24 01:07 rejuvyesh

@rejuvyesh Could you please hardcode the privileged flag, run dstack server from sources, and see if this fixes the problem with GCSFuse?

If all works (it's is important to ensure this in advance), then we can find a way to make privileged configurable from the YAML configuration.

peterschmidt85 avatar Jul 08 '24 09:07 peterschmidt85

This issue is stale because it has been open for 30 days with no activity.

peterschmidt85 avatar Aug 08 '24 01:08 peterschmidt85

Sorry, got busy with other things. Will likely have more time after next week.

rejuvyesh avatar Aug 09 '24 00:08 rejuvyesh

The privileged property is available since 0.18.18. If you still have issues with privileged: true in the run configuration, please open another issue with steps to reproduce.

un-def avatar Oct 18 '24 14:10 un-def