dvc.org icon indicating copy to clipboard operation
dvc.org copied to clipboard

guide: using NFS as a remote storage

Open efiop opened this issue 6 years ago • 7 comments

I.e. mount it to some dir and specify it as a local dir repository for dvc.

efiop avatar Oct 30 '18 14:10 efiop

I have generalized this as "Mounted DVC Storage", which includes not only NFS, but also SSHFS, Samba, or any other data storage that can be network-mounted on a local directory.

I have also built an interactive example that explains how to do it for SSHFS (since this is easier to set up for an interactive example): https://katacoda.com/dvc/courses/examples/mounted-storage

dashohoxha avatar Nov 12 '19 07:11 dashohoxha

@efiop did you mean in https://dvc.org/doc/command-reference/remote/add though? Or somewhere else.

jorgeorpinel avatar Nov 12 '19 20:11 jorgeorpinel

@jorgeorpinel I guess a note there would also be pretty nice as a way of using any storage that you can mount to your local fs.

efiop avatar Nov 15 '19 03:11 efiop

OK. So while #784 greatly simplifies this issue, it won't completely solve it. I'm updating the PR's description to remove the link. Or if you prefer to also add the note Ruslan suggests Dashamir, please reestablish the link to close this issue automatically from your PR. 🙂

jorgeorpinel avatar Nov 18 '19 02:11 jorgeorpinel

I read the comments in #784. Is there anything that should be done here?

iesahin avatar Mar 14 '21 14:03 iesahin

It feels that we don't have a perfect place at the moment to put this note. I think there should be something about remotes in the User Guide in the first place. Starting from basic concepts, expanding into a proper section, where we can put a note in the introductory part about "Unsupported directly remotes" with a solutions like sshfs, mounted NFS, etc. For now, we can probably create a section and a note in the existing remote add, remote modify, remote index with the same meaning.

shcheklein avatar Mar 14 '21 17:03 shcheklein

there should be something about remotes in the User Guide in the first place

Added this to https://github.com/iterative/dvc.org/issues/2866.

jorgeorpinel avatar Oct 12 '21 01:10 jorgeorpinel

QQ @dberenbaum: This has a low priority now but https://github.com/iterative/dvc.org/issues/4303 (part of this one) a high priority. 🤔 Does that mean you want to focus on the latter without addressing the former (this one) ?

jorgeorpinel avatar Mar 11 '23 04:03 jorgeorpinel

@jorgeorpinel In https://dvc.org/doc/user-guide/data-management/remote-storage#file-systems-local-remotes, you added:

You can also use system directories, mounted drives, network resources e.g. network-attached storage (NAS), and other external devices as storage.

Is that enough to close this one? Do we need anything else for this issue specifically?

dberenbaum avatar Mar 11 '23 13:03 dberenbaum

Maybe just add "mounted drives (e.g. NFS)" (link to https://nfs.sourceforge.net/ ?) unless the intention here was specifically about NFS for some reason, and not in general any mounted drives. Note that there are NFS-specific notes in the config options state.dir and index.dir.

jorgeorpinel avatar Mar 12 '23 11:03 jorgeorpinel