New --links parameter not working with mount?

[ErAzOr2k]

What is the problem you are having with rclone?

I uploaded some symlinks with "rclone copy" and parameter "--links" to my gdrive remote. As expected, files with the extension "rclonelink" are generated.

I now expect that the rclonelink files are not visible via rclone mount. Instead they should be translated to symlinks. But actually it is not the case. Only plain rclonelink files are visible.

Additionally I'm also not able to create symlink. Instead I get I/O error (see below).

What is your rclone version (output from rclone version)

rclone v1.46-DEV (latest commit as of today)

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Linux x64

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone --config /config/rclone.conf mount gdrive_decrypted:/ /mnt --allow-other --allow-non-empty --uid=1000 --gid=1000 --umask=007 --links --fast-list --rc--verbose

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

ln -s /tmp/testfile testlink ln: testlink: I/O error

[ErAzOr2k]

Ok if I undetstand it correctly, it only works with local remote. Is it possible to implement this feature for drive remote?

[ncw]

I think this would have to be a new flag for mount, say --use-links or something like that which inteprets the .rclonelink files and turns them back into symlinks.

Therefore you would be able to mount your drive remote, and have the symlinks appear as symlinks.

Is that what is required do you think?

[ncw]

This probably wouldn't be too tricky if someone wanted to have a go?

[ErAzOr2k]

I think this would have to be a new flag for mount, say --use-links or something like that which inteprets the .rclonelink files and turns them back into symlinks.

Therefore you would be able to mount your drive remote, and have the symlinks appear as symlinks.

Is that what is required do you think?

yes, this is exactly the expected behavior :)

[felisucoibi]

I hope this i implemented soon, i would like too mount to follow symlinks instead of show .rclonesymlink and e able to reate symlinks in the cloud (internally .rlclonelink) but for mount a symbolik one.

[diamondsw]

I was just wondering about this myself. It would certainly be useful if the mount layer could seamlessly translate rclonelink files back into symlinks (and bonus points if creating symlinks on a rw rclone mount would create rclonelink files behind the scenes).

[thekiefs]

I would also love this feature!

[kocane]

@ncw Any plans on implementing this? would be very useful. Had plans on using 'UrBackup' with Google Drive mount but as it's built completely on symlinks, it don't seem possible :-(

[mikwaluk]

It would be great if you could add this feature! :)

[dinoboy197]

I think this would have to be a new flag for mount, say --use-links or something like that which inteprets the .rclonelink files and turns them back into symlinks.

Therefore you would be able to mount your drive remote, and have the symlinks appear as symlinks.

Is that what is required do you think?

Hi @ncw - from the documentation for the --links flag in local mode:

--links, -l
...
If you supply this flag then rclone will copy symbolic links from the local storage, and store them as text files, with a '.rclonelink' suffix in the remote storage.
...
This flag applies to all commands.

Considering that the documentation written indicates that the flag applies to all commands, do you think it would reasonable instead to simply allow the pre-existing --links flag to work both for sync as well as mount operations?

[dinoboy197]

I've created a draft PR of a code addition to have the mount command support symbolic links. No extra mount parameters are needed; this works without any extra --links command line arguments.

I still need to add documentation and tests, but it should be ready for use. For anyone who is still interested in this feature, please try out my branch and let me know if that works for you! I hope to have the PR ready for review in the next week or two.

[ncw]

I've created a draft PR of a code addition to have the mount command support symbolic links.

Nice one :-)

No extra mount parameters are needed; this works without any extra --links command line arguments.

I'm not sure about that, but we can discuss more!

I still need to add documentation and tests, but it should be ready for use. For anyone who is still interested in this feature, please try out my branch and let me know if that works for you! I hope to have the PR ready for review in the next week or two.

I note you've done this in the mount layer. There are 3 mount backends - mount/cmount/mount2 and this only applies to one of them. It would be better to do this in the VFS layer which will work with all mount backends. The mount backends will still need a tweak to recognise symlinks though.

[eatnumber1]

For folks following along like myself, the PR is #4933. (thanks for working on this!)

[dinoboy197]

There are 3 mount backends - mount/cmount/mount2 and this only applies to one of them.

Wow - I wondered a bit about those, and didn't realize those were any more than prototypes. Thanks for the note!

It would be better to do this in the VFS layer which will work with all mount backends. The mount backends will still need a tweak to recognise symlinks though.

I doubt that will be any issue, though I admit that I'm not quite sure how the other two mount backends that I didn't tweak actually get enabled. Would you be able to point me to any existing documentation or tips on how to enable the two alternate mount backends so that I can be sure that any changes work with those?

[ncw]

There are 3 mount backends - mount/cmount/mount2 and this only applies to one of them.

Wow - I wondered a bit about those, and didn't realize those were any more than prototypes. Thanks for the note!

No worries!

Note also that the VFS layer is used for the rclone servers too, eg rclone serve http/sftp/webdav

It would be better to do this in the VFS layer which will work with all mount backends. The mount backends will still need a tweak to recognise symlinks though.

I doubt that will be any issue, though I admit that I'm not quite sure how the other two mount backends that I didn't tweak actually get enabled. Would you be able to point me to any existing documentation or tips on how to enable the two alternate mount backends so that I can be sure that any changes work with those?

Sure...

You use rclone mount ..., rclone mount2 ... and rclone cmount ....

You need to supply the build tag -tags cmount to build with cmount support, and you'll need the FUSE headers and a C compiler. cmount uses cgo with the FUSE library, whereas mount talks straight to the kernel.

Note that rclone mount is the default mount for linux/bsd/other whereas rclone cmount is the default mount for Windows/macOS as rclone mount doesn't work there.

At some point I plan to drop mount2 but rclone needs mount and cmount!

[dinoboy197]

After some initial refactoring, I realized that symlink support down into the VFS layer means building symlink support for several other components - not just mount and cmount. I realized that I would need to make some fairly significant design choices regarding how to represent symlinks at that layer - should they be a File, but have a new isSymlink method, or should there be a new type that derives from Node entirely; should symlinks support propagate further down chain into the FS layer, etc. While I was aware earlier that my symlink addition for the mount command was fairly trivial and somewhat superficial, I didn't have a proper appreciation for what all would go into proper support in the VFS layer. Not surprising considering I'm a first-time rclone contributor!

I don't believe I have the right experience or vision to be making these kinds of changes in the VFS layer as a first-time contributor to rclone - I'd have to ask many questions and take up a considerable amount of time trying to sync with the future vision of VFS in order to come up with a solution that I thought was satisfactory. Rather, I'd feel much more comfortable if someone with more of an ownership stake in the VFS architecture and vision implemented the symlink support they'd like to see in that layer, then I'd be comfortable retrofitting my mount solution to use whatever VFS hooks were built out.

I say this not a way to suggest that my changes should go to the rclone mainline just because I'd rather not implement things in the VFS layer. Rather, I do think that VFS support is the right way to implement this and look forward to someone with more experience (both with rclone and with Golang) to drive that a bit. I'm sorry if this comes as a disappointment for anyone waiting for a full solution.

I'll keep an eye out here, and should I become more well-versed with the codebase and VFS direction at a later date, perhaps will pick this up again!

[ivandeex]

The attempt PR #4933 was abandoned and closed. New developers can pick it by the link above and use its patch as a basis.

[pasnox]

I've created a draft PR of a code addition to have the mount command support symbolic links.

Nice one :-)

No extra mount parameters are needed; this works without any extra --links command line arguments.

I'm not sure about that, but we can discuss more!

I still need to add documentation and tests, but it should be ready for use. For anyone who is still interested in this feature, please try out my branch and let me know if that works for you! I hope to have the PR ready for review in the next week or two.

I note you've done this in the mount layer. There are 3 mount backends - mount/cmount/mount2 and this only applies to one of them. It would be better to do this in the VFS layer which will work with all mount backends. The mount backends will still need a tweak to recognise symlinks though.

Hey, I'm trying to try to fix this feature request as it is a very needed feature for me. Unfortunately, I comes from C/C++, and Go is just shadow for me, so i will have to minimal learn it as I need. Would you wanna give some inputs on what/where would be to be patched in the vfs ?

Thanks,

[danielhatton]

On Sun, 13 Feb 2022, Filipe Azevedo wrote:

Hey, I'm trying to try to fix this feature request as it is a very needed feature for me. Unfortunately, I comes from C/C++, and Go is just shadow for me, so i will have to minimal learn it as I need. Would you wanna give some inputs on what/where would be to be patched in the vfs ?

In case the question was addressed to me: I'm not at all familiar with the rclone source code (and indeed I no longer have access to a cloud storage service on which to test anything). Sorry.

[pasnox]

it was for @ncw

[pasnox]

@dinoboy197 What's the status of your fork in regards to mount supporting symlink ?

[dinoboy197]

Last time I rebased my dinoboy197/traack_symlinks branch from rclone master, it brought in commits up through 2022-02-18.

[ncw]

@dinoboy197 do you want to have another go at sending a PR for this? I like the concept. Most of the code you've done in cmd/mount needs to go into the vfs - then it will be usable for all users of the vfs, most importantly cmd/cmount which is how mount works on macOS and Windows.

[dinoboy197]

I appreciate the encouragement! I don't have the time to dedicate to the refactor right now, but it is good to know what the appropriate direction would be if I take this up in the future.

[rkzed94]

really looking forward for this to be implemented.. the only missing feature that i need is symlink support for rclone mount.

[Bra1nsen]

sorry I don't see through here completely. is it now possible to set hardlinks via rclone and gdrive? and if so how?

[ncw]

Hardlinks aren't possible through rclone at the moment.

[0xallie]

I would love to see this for both mount and serve as well, so yeah, it would be best if it was implemented on the VFS layer.

[pasnox]

@ncw I'm trying to implement changes for this feature, but I will need some inputs. Right now i'm struggling to find which part of the code received the remote data that is then used to fill the vfs, so that I can hook the part to add symlinks support. Any idea where I would dig in ?

EDIT: Nevermind, found what I needed

[pasnox]

I got the first bits working for vfs / mount / symlinks - can mount and see / read / open symlinks files on linux. I will continue the work when back from vacations - around July 22th.