attic hangs when initializing repository on a cifs mount

[alinhan]

Hi,

I hit the following problem: when initializing a repository on a cifs mount, attic hangs:

$ sudo attic init ~/Tmp/mnt/Dir1/dir2 Initializing repository at "/home/myuser/Tmp/Dir1/dir2" Encryption NOT enabled. Use the "--encryption=passphrase|keyfile" to enable encryption.

And it stays stuck like this. I left it even half an hour and nothig else happens after the "Use the..." line gets printed on the screen.

I can read/write with sudo the files from the ~/Tmp/mnt mount just fine. I can even backup files with attic from that mount.

I'm using Ubuntu 14.10. I've tried this with attic 0.13 from the official Ubuntu repository, and also with attic 0.14 installed with pip3.

The mount is created like this:

sudo mount -t cifs -o user=my-windows-user //lan-ip-address-of-win7-machine/d$ ~/Tmp/mnt

Please help.

Thanks, Alin.

Edit: I had to correct a couple of commands because they were showing incorrectly.

[Sedins]

I'm not in the need of creating archives on CIFS myself, but tried this anyhow since I have the environment set up for it.

Using a Windows 2008 R2 Server with a Windows share and initialized an repository as specified in the first post. The init-phase just gets stuck forever. When aborting and checking the directory afterwards I can see that there are some files created (config, README and an empty data-folder).

Attic fails with message "Error: Object with key b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' not found in repository test.attic" if creating an archive to this repository. Not particularly strange, as the init part was previously incomplete.

Tried a workaround. Initialized the repository directly on an ext4 drive, which worked, whereas I moved the repository to the Windows share. So far so good, but when executing "attic create", it gets stuck forever again.

attic create test.attic::test-1 /home/user/ Initializing cache...

Edit: The user has permissions needed to read, write, delete, change files and folder in the destination. Share has full control for everyone and the NTFS permissions is modify for this particular user.

Edit: Tests were performed from a Debian Wheezy machine with Attic 0.14 installed with pip3.2.

[ThomasWaldmann]

The key 0 is the manifest, which is needed pretty much as the first thing everywhere.

So we need to find out why init is stuck...

[eht16]

I just got the same issue. Similar setup to @alinhan: Ubuntu 14.10 with attic 0.13 from the repos and a CIFS mount as remote storage (I have no better choice for my workstation at work).

attic init just hung on the CIFS mount while it created a repository successfully on local disk. Then I ran 'attic create' with strace and got the following results (boring, not relevant details stripped):

execve("/usr/bin/attic", ["attic", "init", "/mnt/remote/backup.attic"], [/* 57 vars */]) = 0
brk(0)                                  = 0x10da000
[...]
stat("/mnt/remote/backup.attic", 0x7fff38e51e00) = -1 ENOENT (No such file or directory)
stat("/mnt/remote/backup.attic", 0x7fff38e51e00) = -1 ENOENT (No such file or directory)
mkdir("/mnt/remote/backup.attic", 0777)   = 0
open("/mnt/remote/backup.attic/README", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 4
fstat(4, {st_mode=S_IFREG|0755, st_size=0, ...}) = 0
ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff38e51c00) = -1 ENOTTY (Inappropriate ioctl for device)
fstat(4, {st_mode=S_IFREG|0755, st_size=0, ...}) = 0
lseek(4, 0, SEEK_CUR)                   = 0
ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff38e51b50) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(4, 0, SEEK_CUR)                   = 0
lseek(4, 0, SEEK_CUR)                   = 0
write(4, "This is an Attic repository\n", 28) = 28
close(4)                                = 0
mkdir("/mnt/remote/backup.attic/data", 0777) = 0
fstat(3, {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 9), ...}) = 0
read(3, "X\363\214'Z\267\224\326\350\23\350\337 \211\252\360\321\332\344$*R,\2270\r\301`L\232V3", 32) = 32
open("/mnt/remote/backup.attic/config", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 4
fstat(4, {st_mode=S_IFREG|0755, st_size=0, ...}) = 0
ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff38e51c00) = -1 ENOTTY (Inappropriate ioctl for device)
fstat(4, {st_mode=S_IFREG|0755, st_size=0, ...}) = 0
lseek(4, 0, SEEK_CUR)                   = 0
ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff38e51b50) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(4, 0, SEEK_CUR)                   = 0
lseek(4, 0, SEEK_CUR)                   = 0
write(4, "[repository]\nversion = 1\nsegment"..., 148) = 148
close(4)                                = 0
stat("/mnt/remote/backup.attic", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
open("/mnt/remote/backup.attic/config", O_RDONLY|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFREG|0755, st_size=148, ...}) = 0
ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff38e51a10) = -1 ENOTTY (Inappropriate ioctl for device)
fstat(4, {st_mode=S_IFREG|0755, st_size=148, ...}) = 0
lseek(4, 0, SEEK_CUR)                   = 0
ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff38e51960) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(4, 0, SEEK_CUR)                   = 0
read(4, "[repository]\nversion = 1\nsegment"..., 8192) = 148
read(4, "", 8192)                       = 0
close(4)                                = 0
open("/mnt/remote/backup.attic/config", O_RDWR|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFREG|0755, st_size=148, ...}) = 0
ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff38e51990) = -1 ENOTTY (Inappropriate ioctl for device)
fstat(4, {st_mode=S_IFREG|0755, st_size=148, ...}) = 0
lseek(4, 0, SEEK_CUR)                   = 0
lseek(4, 0, SEEK_CUR)                   = 0
ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff38e518e0) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(4, 0, SEEK_CUR)                   = 0
fcntl(4, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) = 0
write(1, "Encryption NOT enabled.\nUse the "..., 87) = 87
write(1, "\n", 1)                       = 1
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f03515e9000
openat(AT_FDCWD, "/mnt/remote/backup.attic", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 5
getdents(5, /* 5 entries */, 32768)     = 136
getdents(5, /* 0 entries */, 32768)     = 0
close(5)                                = 0
openat(AT_FDCWD, "/mnt/remote/backup.attic/data", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 5
getdents(5, /* 2 entries */, 32768)     = 48
getdents(5, /* 0 entries */, 32768)     = 0
close(5)                                = 0
openat(AT_FDCWD, "/mnt/remote/backup.attic", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 5
getdents(5, /* 5 entries */, 32768)     = 136
getdents(5, /* 0 entries */, 32768)     = 0
close(5)                                = 0
fcntl(4, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}Process 32313 detached
 <detached ...>
attic: Error: Keyboard interrupt

The third last line is where attic hangs and before I cancelled the process with Ctrl-C. I don't know exactly what kind of lock attic is trying to do but obviously F_WRLCK is related to some locking. So I assume the hang is related to locking which is either not supported in that way by CIFS at all or not by certain server implementations. Not sure because I don't know enough about CIFS and the various server implementations. In my case, the server is some Windows 2008 or 2012 server which acts as file server.

I then read the mapage of mount.cifs to check if there are any locking-related options and actually there is: "byte range lock". The manpage says:

nobrl
           Do not send byte range lock requests to the server. This is necessary for certain applications that break with cifs style mandatory byte range locks (and most cifs servers do not yet
           support requesting advisory byte range locks).

Adding this option to the mount options in /etc/fstab (or however you mount your storage) and remounting the storage, enables attic to create a repository and successfully perform backups.

HTH.

[Sedins]

Just wanted to add that I've tested to mount with the setting given by eht16 and can confirm that Attic is able to initialize archives and create backups on CIFS shares now.

[ThomasWaldmann]

The remaining question is just whether locking still works as intended.

[opsydev]

I am experiencing the same thing. Attic hangs at

fcntl(4, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}

Setting the nobrl flag does seem to fix the issue. I can test the lock further if you can provide information about what I am supposed to look for.

[ThomasWaldmann]

@alinhan @eht16 @twistedR @Sedins ^^^ can you test that?

[eht16]

I'd like to test it but the patch from https://github.com/borgbackup/borg/pull/94 doesn't apply to Attic 0.16. Usually, I would try borgbackup and/or fiddle with the patch. But I can test this only at work and there I can't spend so much time on this :(. @ThomasWaldmann could you maybe provide a patch for Attic 0.16?

[ThomasWaldmann]

Sorry to say, but attic and borg diverged here a bit (e.g. the new locking code tests use py.test, a mock issue showed up and was fixed, etc.) - making a patch would be a bit more work than usual.

Also, I am not sure if it would be acceptable for attic, there is quite a backlog of not accepted (yet?) pull requests.

BTW, after writing above post, I did some practical tests with a CIFS mount myself and it worked.

[eht16]

Ok, I was just confused because you ask for testing a patch for borg on an issue on attic.

[larsmm]

When will there be a fix? I have the problem everywhere: debian 8.1 --> qnap nas, debian 8.1 --> windows 7. "nobrl" in mount options does not help. On the shares everyone can do everything. At the moment I see no way to backup to Windows. I tested attic 1.6 and the newest from git on python 3.4.2. Attic init worked, attic create not.

Backup command: attic create --stats /mnt/backup_qnap/backup.attic::server-'date +%Y-%m-%d' /mnt/ --exclude /mnt/backup_qnap

The error: attic: Error: Object with key b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' not found in repository /mnt/backup_qnap/backup.attic

My mount command: mount -t cifs "//qnap/public" /mnt/backup_qnap -o user=...,passwd=...,nobrl Please show me the mount command that worked.

[tonyarkles]

I am currently experiencing the same problem. I waited for... however long it took to have a bath. Long enough :)

[0xjairo]

I'm experiencing the same issue with my CIFS mount to a QNAP NAS.

attic help prints version 0.16

Some files are created with the init command, but attic hangs while initializing.

tree output of the attic repository:

$ tree
.
├── config
├── data
└── README

1 directory, 2 files

The ./data/ directory is empty

$ cat config 
[repository]
version = 1
segments_per_dir = 10000
max_segment_size = 5242880
id = 8cc2a93df9212bfbae11b23b9d2bbba5f8239dd6e1c4354e4c377ea4ecb16bf2
$ cat README 
This is an Attic repository