genie icon indicating copy to clipboard operation
genie copied to clipboard

Rclone doesn't mount properly when genie is running

Open codenamecarterx opened this issue 4 years ago • 4 comments

Hello, this may be an easy fix on my part as I am completely new to linux. I installed genie so that I can auto mount my google drive using rclone. I was able to get rclone.service running using systemd without any errors being returned. However, the mounted drive is completely empty. The files in google drive don't show up. This only happens after running the genie -s command. For example, when manually running the command "rclone mount gdrive: /GD" before initiating genie the results are as expected. I can successfully browse the contents of my gdrive in the /GD directory. Once genie has started, running the same command (manually or auto via rclone.service) results in an empty /GD directory. Any idea on how to fix this?

codenamecarterx avatar Mar 15 '21 18:03 codenamecarterx

Hm.

Well, the first thing I can tell you, unhelpful as it alone may be, is that it can be made to work, 'cause I just set up rclone against my Google Drive, mounting via systemd via genie, and it works fine with the files showing up. So FWIW, at least you aren't pursuing something that can't be made to work.

That said, I don't have much of an idea as to why yours doesn't, under these circumstances. I'm think I'm going to need a lot more by way of details - output of id inside and outside the bottle, copy of your ~/.config/rclone/rclone.conf (feel free to redact client secrets and access tokens), your distro, how and from where you installed rclone, copy of your systemd service file for rclone, permissions of the mount point folder, any output from systemctl status rclone and from a manual rclone mount, output from mount once the folder is mounted, that sort of thing. And, just to be sure, you're mounting the drive inside genie without having previously mounted the drive outside genie, right?

cerebrate avatar Mar 15 '21 19:03 cerebrate

Thanks for the fast response! It is great to hear that it can work as after countless hours of trying and not being able to find any guides for what I've been trying to do, I was beginning to lose hope : ). Here is most of the info you requested below.

Distro > Ubuntu 20.04 LTS Focal ID inside > plex@DESKTOP-9BHFDU3:~$ genie -s Waiting for systemd....!! plex@DESKTOP-9BHFDU3-wsl:~$ id uid=1000(plex) gid=1000(plex) groups=1000(plex),4(adm),20(dialout),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),117(netdev),1001(docker) ID outside > uid=1000(plex) gid=1000(plex) groups=1000(plex),4(adm),20(dialout),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),117(netdev),1001(docker)

/GD permissions > drwxrwxr-x 1 plex plex 0 Mar 15 16:13 /GD

rclone config `[gdrive] type = drive client_id = 1024867742093-0p8fllihsitie72tg350jrdamjubtb49.apps.googleusercontent.com client_secret = xxxxxxxxxxxxxxxxxxx scope = drive token = xxxxxxxxxxxxxxxxxxx

[gdrivecrypt] type = crypt remote = gdrive:Media filename_encryption = standard directory_name_encryption = true password = qRshSQvoIFEgV0H_H355qKMCd9tzd_SxSJIBUQQ6 password2 = dvrc_ws3IGF123h6zOo2geV8RdLMNOP-3PY

[merged] type = union upstreams = G: F: P: E: create_policy = ff action_policy = ff`

rclone.service

[Unit]
Description=RClone Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/plex/.config/rclone/rclone.conf
KillMode=none
RestartSec=5
ExecStart=/usr/bin/rclone mount gdrivecrypt: /GD \
# This is for allowing users other than the user running rclone access to the mount
--allow-other \
# Google Drive is a polling remote so this value can be set very high and any changes are detected via polling.
--dir-cache-time 1000h \
# The log level output 
-vv \
# Location of the log file
--log-file /opt/rclone/logs/rclone.log \
# I reduce the poll interval down to 15 seconds as this makes changes appear fast the API quotas per day are huge
--poll-interval 10s \
# This is setting the file permission on the mount to user and group have the same access and other can read
--umask 002 \
# Please set this to your own value below
--user-agent desktopapp \
# This sets up the remote control daemon so you can issue rc commands locally
--rc \
# This is the default port it runs on
--rc-addr :5572 \
# no-auth is used as no one else uses my server and it is not a shared seedbox
--rc-no-auth \
# The local disk used for caching
--cache-dir=/mnt/a/cache \
# This is used for caching files to local disk for streaming
--vfs-cache-mode writes \
# This limits the cache size to the value below
--vfs-cache-max-size 500G \
# This limits the age in the cache if the size is reached and it removes the oldest files first
--vfs-cache-max-age 336h \
# This sets a per file bandwidth control and I limit this to a little bigger than my largest bitrate I'd want to play
--bwlimit-file=52M
ExecStop=/bin/fusermount -uz /GD
ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --rc-addr localhost:5572 _async=true
Restart=on-failure
User=plex


[Install]
WantedBy=multi-user.target

Used the command from the rclone site to install curl https://rclone.org/install.sh | sudo bash. It is located in its default directory /usr/bin/rclone

Yes, I am mounting it inside genie only. I did make that mistake early on but the service would just fail with errors.

codenamecarterx avatar Mar 15 '21 21:03 codenamecarterx

Here is what my log looks like after manually running rclone mount outside of genie. As you can see, the contents of my gdrive can actually be seen.

2021/03/15 16:05:59 DEBUG : Using config file from "/home/plex/.config/rclone/rclone.conf" 2021/03/15 16:05:59 DEBUG : rclone: Version "v1.55.0-beta.5270.3cf6ea848" starting with parameters ["rclone" "mount" "gdrivecrypt:" "/GD" "--vfs-cache-mode" "writes" "--allow-other" "--log-file" "/opt/rclone/logs/rclone.log" "-vv"] 2021/03/15 16:05:59 DEBUG : Creating backend with remote "gdrivecrypt:" 2021/03/15 16:05:59 DEBUG : Creating backend with remote "gdrive:Media" 2021/03/15 16:06:00 DEBUG : Google drive root 'Media': root_folder_id = "0AEvj48jRyrFwUk9PVA" - save this in the config to speed up startup 2021/03/15 16:06:00 DEBUG : vfs cache: root is "/home/plex/.cache/rclone/vfs/gdrivecrypt" 2021/03/15 16:06:00 DEBUG : vfs cache: metadata root is "/home/plex/.cache/rclone/vfs/gdrivecrypt" 2021/03/15 16:06:00 DEBUG : Creating backend with remote "/home/plex/.cache/rclone/vfs/gdrivecrypt" 2021/03/15 16:06:00 DEBUG : Encrypted drive 'gdrivecrypt:': Mounting on "/GD" 2021/03/15 16:06:00 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2021/03/15 16:06:00 DEBUG : : Root: 2021/03/15 16:06:00 DEBUG : : >Root: node=/, err= 2021/03/15 16:06:20 DEBUG : /: Attr: 2021/03/15 16:06:20 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:23 DEBUG : /: Attr: 2021/03/15 16:06:23 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:23 DEBUG : /: ReadDirAll: 2021/03/15 16:06:23 DEBUG : /: >ReadDirAll: item=4, err= 2021/03/15 16:06:23 DEBUG : /: Attr: 2021/03/15 16:06:23 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:23 DEBUG : /: Lookup: name="Media" 2021/03/15 16:06:23 DEBUG : /: >Lookup: node=Media/, err= 2021/03/15 16:06:23 DEBUG : Media/: Attr: 2021/03/15 16:06:23 DEBUG : Media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:23 DEBUG : /: Lookup: name="System Volume Information" 2021/03/15 16:06:23 DEBUG : /: >Lookup: node=System Volume Information/, err= 2021/03/15 16:06:23 DEBUG : System Volume Information/: Attr: 2021/03/15 16:06:23 DEBUG : System Volume Information/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:23 DEBUG : /: Attr: 2021/03/15 16:06:23 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:23 DEBUG : Media/: ReadDirAll:

2021/03/15 16:06:31 DEBUG : Media/Movies/: >ReadDirAll: item=4, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/: Attr: 2021/03/15 16:06:31 DEBUG : Media/Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/: Lookup: name="4K" 2021/03/15 16:06:31 DEBUG : Media/Movies/: >Lookup: node=Media/Movies/4K/, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/4K/: Attr: 2021/03/15 16:06:31 DEBUG : Media/Movies/4K/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/: Lookup: name="HD" 2021/03/15 16:06:31 DEBUG : Media/Movies/: >Lookup: node=Media/Movies/HD/, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/HD/: Attr: 2021/03/15 16:06:31 DEBUG : Media/Movies/HD/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/: Attr: 2021/03/15 16:06:31 DEBUG : Media/Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/: ReadDirAll: 2021/03/15 16:06:31 DEBUG : Media/Movies/: >ReadDirAll: item=4, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/: Attr: 2021/03/15 16:06:31 DEBUG : Media/Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/: Attr: 2021/03/15 16:06:31 DEBUG : Media/Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/: Attr: 2021/03/15 16:06:31 DEBUG : Media/Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:31 DEBUG : Media/Movies/4K/: ReadDirAll: 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/: >ReadDirAll: item=21, err= 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/: Attr: 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/: Lookup: name=".plexignore" 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/: >Lookup: node=Media/Movies/4K/.plexignore, err= 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/.plexignore: Attr: 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/.plexignore: >Attr: a=valid=1s ino=0 size=114 mode=-rw-r--r--, err= 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/: Lookup: name="Black Panther (2018)" 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/: >Lookup: node=Media/Movies/4K/Black Panther (2018)/, err= 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/Black Panther (2018)/: Attr: 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/Black Panther (2018)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err= 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/: Lookup: name="Deadpool (2016)" 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/: >Lookup: node=Media/Movies/4K/Deadpool (2016)/, err= 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/Deadpool (2016)/: Attr: 2021/03/15 16:06:32 DEBUG : Media/Movies/4K/Deadpool (2016)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=

Here's what the log looks like when running rclone mount inside of genie. Google drive root 'Media': Checking for changes on remote just repeats over and over again but the subdirectories never load.

2021/03/15 16:13:08 DEBUG : Using config file from "/home/plex/.config/rclone/rclone.conf" 2021/03/15 16:13:08 DEBUG : rclone: Version "v1.55.0-beta.5270.3cf6ea848" starting with parameters ["/usr/bin/rclone" "mount" "gdrivecrypt:" "/GD" "--allow-other" "--dir-cache-time" "1000h" "-vv" "--log-file" "/opt/rclone/logs/rclone.log" "--poll-interval" "10s" "--umask" "002" "--user-agent" "desktopapp" "--rc" "--rc-addr" ":5572" "--rc-no-auth" "--cache-dir=/mnt/a/cache" "--vfs-cache-mode" "writes" "--vfs-cache-max-size" "500G" "--vfs-cache-max-age" "336h" "--bwlimit-file=52M"] 2021/03/15 16:13:08 NOTICE: Serving remote control on http://[::]:5572/ 2021/03/15 16:13:08 DEBUG : Creating backend with remote "gdrivecrypt:" 2021/03/15 16:13:08 DEBUG : Creating backend with remote "gdrive:Media" 2021/03/15 16:13:09 DEBUG : Google drive root 'Media': root_folder_id = "0AEvj48jRyrFwUk9PVA" - save this in the config to speed up startup 2021/03/15 16:13:09 DEBUG : vfs cache: root is "/mnt/a/cache/vfs/gdrivecrypt" 2021/03/15 16:13:09 DEBUG : vfs cache: metadata root is "/mnt/a/cache/vfs/gdrivecrypt" 2021/03/15 16:13:09 DEBUG : Creating backend with remote "/mnt/a/cache/vfs/gdrivecrypt" 2021/03/15 16:13:09 DEBUG : Encrypted drive 'gdrivecrypt:': Mounting on "/GD" 2021/03/15 16:13:09 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2021/03/15 16:13:09 DEBUG : : Root: 2021/03/15 16:13:09 DEBUG : : >Root: node=/, err= 2021/03/15 16:13:09 DEBUG : rc: "vfs/refresh": with parameters map[_async:true recursive:true] 2021/03/15 16:13:09 DEBUG : rc: "vfs/refresh": reply map[jobid:1]: 2021/03/15 16:13:09 DEBUG : : Reading directory tree 2021/03/15 16:13:15 DEBUG : : Reading directory tree done in 6.36184903s 2021/03/15 16:13:19 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:13:29 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:13:39 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:13:49 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:13:59 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:14:09 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:14:09 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2021/03/15 16:14:19 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:14:29 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:14:39 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:14:49 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:14:59 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:15:09 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:15:09 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0) 2021/03/15 16:15:19 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:15:29 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:15:39 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:15:49 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:15:59 DEBUG : Google drive root 'Media': Checking for changes on remote 2021/03/15 16:16:09 DEBUG : Google drive root 'Media': Checking for changes on remote

systemctl status rclone when genie is off says this

System has not been booted with systemd as init system (PID 1). Can't operate. Failed to connect to bus: Host is down

Inside genie systemctl status rclone says `● rclone.service - RClone Service Loaded: loaded (/etc/systemd/system/rclone.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2021-03-15 16:13:09 CDT; 57min ago Process: 331 ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --rc-addr localhost:5572 _async=true (code=exited, status=0/SUCCESS) Main PID: 249 (rclone) Status: "[17:10] vfs cache: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)" Tasks: 12 (limit: 4714) Memory: 53.1M CGroup: /system.slice/rclone.service └─249 /usr/bin/rclone mount gdrivecrypt: /GD --allow-other --dir-cache-time 1000h -vv --log-file /opt/rclone/logs/rclone.log --poll-interval 10s --umask 002 --user-agent desktopapp --rc --rc-addr :5572 --rc-no-auth --cache-dir=/mnt/a/cache --vfs-cache-mode writes --vfs-cache-max-size 500G --vfs-cache-max-age 336h --bwlimit-file=52M

Mar 15 16:13:08 DESKTOP-9BHFDU3-wsl systemd[1]: Starting RClone Service... Mar 15 16:13:09 DESKTOP-9BHFDU3-wsl rclone[331]: { Mar 15 16:13:09 DESKTOP-9BHFDU3-wsl rclone[331]: "jobid": 1 Mar 15 16:13:09 DESKTOP-9BHFDU3-wsl rclone[331]: } Mar 15 16:13:09 DESKTOP-9BHFDU3-wsl systemd[1]: Started RClone Service. `

codenamecarterx avatar Mar 15 '21 22:03 codenamecarterx

There has been no activity on this issue for over 60 days. If there is no further activity within 7 days from this message, it will be automatically closed.

github-actions[bot] avatar May 15 '21 02:05 github-actions[bot]