borg under windows 10's linux subsystem

[ThomasWaldmann]

try it, test it, document outcome.

[robaato]

First impression: borg 1.0.8 from bundled binary borg-linux64 -> starts fine

mkdir aaa ./borg-linux64 init aaa ./borg-linux64 create --info --progress aaa::bbb /usr/ ./borg-linux64 list aaa::bbb ./borg-linux64 check aaa ./borg-linux64 extract aaa::bbb diff -rN --brief /usr usr
// no result except for few broken links, which are already broken on source

all above run fine. Looks promising. Will try to backup and restore something from Windows partitions, above is all inside linux subsystem.

[ThomasWaldmann]

@robaato thanks for the update. btw, you don't need to create the repo dir first, borg init does that, too.

[ThomasWaldmann]

My first (install via pip from pypi package) and second impression (install from git repo) is also that it works.

Installation is like on (native) Ubuntu (Trusty, 14.04). Only thing I noticed not working is the FUSE support (borg mount) as it requires the fuse kernel module, which is not there.

Now running the tests...

[ThomasWaldmann]

23 test fails:

• create_test_files: os.mknod -> Function not implemented (ENOSYS) (with root it works, so just a bad errno?) - solved by #1963.
• get_acl (from linux platform module) returns dict without b'acl_access' key (no ACL support?)
• fuse tests fail (no fuse kernel support) - likely can be avoided by omitting llfuse package.

The failing ones are somehow expected, all others work.

[ThomasWaldmann]

Docs updated, see PR #1962.

[BluCodeGH]

I have run borg 1.0.9 under the windows 10 subsystem as an administrator and while using sudo. I am backing up files from my (windows) home folder to another drive. The only errors I am seeing are [Errno 13] Permission Denied, and [Errno 5] Input/output error. Error 13 occurs on some files and folders, such as AppData/Local/ElevatedDiagnostics or Appdata/Local/Google/Chrome/User Data/Default/Current Tabs, and error 5 occures on files called LOCK, in various folders.

[enkore]

That's nice to hear.

Locking errors on Windows are to be expected unless VSS is used, although EIO / I/O error is kind of a dramatic error code for that.

[BluCodeGH]

Is there a way to get borg on lxss working with VSS?

[jdchristensen]

And any idea why some files (like ElevatedDiagnostics and Current Tabs) get Errno 13, while the LOCK files get Errno 5? The LOCK files are probably not important, but it would be good if the other files could be backed up.

[PythonNut]

@BluCodeGH

Is there a way to get borg on lxss working with VSS?

I just gave this a shot, and as far as I can tell, the answer is no, since WSL doesn't support NTFS Junctions, which is how VSS snapshots are "mounted", which is unfortunate.

[Clete2]

I am running under Windows 10 subsystem backing up 1.5TB of data without any errors. borg check on the remote host is also reporting that the integrity is good. Is there anything else I should check to ensure that the backup is complete and accurate?

FYI -- I am backing up video game downloads from the following game downloading services: Steam, Blizzard (formerly Battle.net), Origin, and some other misc games without downloading services.

[ThomasWaldmann]

@Clete2 borg check (in full mode, not just in --repository-only mode) being ok is quite a good sign already.

a more practical test is to actually extract all data again and compare it to the original data. or extract --dry-run if you do not have the storage space.

[mnpenner]

Assuming Borg works under WSL/Win10Bash, how would we get it to start automatically on Windows startup?

[enkore]

Perhaps this helps: https://superuser.com/a/1112022/107877

[ThomasWaldmann]

Well, maybe a matter of taste, but I find anything major starting automatically on startup (or shutdown) rather annoying.

[mnpenner]

@ThomasWaldmann I do too, but we're talking about backup here. That has to run continuously, or we're vulnerable, no?

Thanks @enkore I'll take a look.

[sdellenb]

😃 Backup works surprisingly well with WSL, though there's not much data yet in the user's Windows folders. I've configured the Windows Task Scheduler to run bash (C:\Windows\System32\bash.exe) with a wrapper script (arguments -c "~/bin/borgmatic_wrapper.sh") daily at 18:00, with the option "Run task as soon as posible after a schedule start is missed". I'll see how the setup holds up in the future.

Note: To get borgbackup 1.0.10, I had to install it from an alternative PPA. I also installed borgmatic for easier configuration and my wrapper script collects the borgmatic output and mails it with msmtp (on Linux machines, I let cron handle this).

[rbn920]

I used the linux64 binary today and seems to work just fine.

[theChaosCoder]

I just wanted to report that you can now mount network shares under WSL with the new Windows 10 Fall Creators Update. Just tested it and it's working great. No more cygwin needed.

$ sudo mount -t drvfs '\\server\share' /mnt/share

See also https://blogs.msdn.microsoft.com/wsl/2017/04/18/file-system-improvements-to-the-windows-subsystem-for-linux/

[theChaosCoder]

I need to correct myself:

it worked with my small repo. But I get often this error:

Local Exception
Traceback (most recent call last):
  File "borg/archiver.py", line 4024, in main
  File "borg/archiver.py", line 3952, in run
  File "borg/archiver.py", line 148, in wrapper
  File "borg/archiver.py", line 528, in do_create
  File "borg/archiver.py", line 494, in create_inner
  File "borg/archive.py", line 446, in save
  File "borg/archive.py", line 250, in flush
  File "borg/archive.py", line 266, in write_chunk
  File "borg/cache.py", line 872, in add_chunk
  File "borg/crypto/key.py", line 362, in encrypt
  File "borg/crypto/nonces.py", line 85, in ensure_reservation
  File "borg/crypto/nonces.py", line 48, in commit_repo_nonce_reservation
  File "borg/repository.py", line 309, in commit_nonce_reservation
  File "borg/platform/base.py", line 170, in __exit__
  File "borg/platform/base.py", line 138, in close
  File "borg/platform/base.py", line 58, in sync_dir
OSError: [Errno 5] Input/output error

Platform: Linux Chaos-PC 4.4.0-43-Microsoft #1-Microsoft Wed Dec 31 14:42:53 PST 2014 x86_64 x86_64
Linux: debian stretch/sid
Borg: 1.1.0  Python: CPython 3.5.4
PID: 47  CWD: /mnt/borgBackup

Even borg init triggers OSError: [Errno 5] Input/output error.

I remember having similar problems with borg 1.0.10 in cygwin, but 1.0.9 was ok. I will investigate further.

[ThomasWaldmann]

Often or always? Maybe syncing a dir is not supported on windows?

[theChaosCoder]

always borg create this error randomly (large repo), and this error is produced always at the end. (tested also with small repos)

[theChaosCoder]

Is there an easy way to test sync?

[ThomasWaldmann]

import errno
import os
import sys

def sync_dir(path):
    fd = os.open(path, os.O_RDONLY)
    try:
        os.fsync(fd)
    except OSError as os_error:
        # Some network filesystems don't support this and fail with EINVAL.
        # Other error codes (e.g. EIO) shouldn't be silenced.
        if os_error.errno != errno.EINVAL:
            raise
    finally:
        os.close(fd)

if __name__ == '__main__':
    path = sys.argv[1]
    print("Syncing dir %s" % path)
    sync_dir(path)

Usage:

python3 syncdir.py somedirectory

[theChaosCoder]

Syncing dir /mnt/borgBackup/
Traceback (most recent call last):
  File "stest.py", line 20, in <module>
    sync_dir(path)
  File "stest.py", line 8, in sync_dir
    os.fsync(fd)
OSError: [Errno 5] Input/output error

/mnt/borgBackup/ fails <-- mounted via the new DrvFs (smb share) /mnt/d/ is OK! /mnt/d/ == D:\

[ThomasWaldmann]

so, the "linux filesystem" syncs dirs ok (in WSL), but the mounted drvfs always fails with the dir sync?

[theChaosCoder]

exactly.

[theChaosCoder]

Also a mounted usb stick works too

sudo mount -t drvfs D: /mnt/Ddrive/ <-- sync OK sudo mount -t drvfs F: /mnt/usbstick/ <-- sync OK sudo mount -t drvfs '\\synology\Backups' /mnt/borgBackup/ <-- sync Fail (also fails if mounted in Win as a Network Drive z: )

//edit Dir sync via cygwin works with the samba share.

[ThomasWaldmann]

\synology\Backups is what? smbfs via drvfs?

Guess that's a bug in one of these filesystems. They should either support dir sync or say "invalid" (EINVAL), but not "I/O error" (EIO).

[theChaosCoder]

yep it is smbfs via drvfs I also thinks it's a bug or just not implemented...