As root: "You need to run this command as root."

[HaleTom]

As root, I'm being told I need to be root:

# id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),19(log)
# bedup dedup
Skipped 17 frozen volumes in filesystem <home>
Skipped 15 frozen volumes in filesystem <rootfs>
Not scanning /mnt/btrfs-vol/home, generation is still 383714
Scanning volume /home generations from 585948 to 585949, with size cutoff 1
00.02 Scanned 4172 retained 4
Not scanning /mnt/btrfs-vol/home/btrbk-snap, generation is still 585179
Not scanning /home/var, generation is still 585179
Not scanning /mnt/btrfs-vol/rootfs, generation is still 13965
Not scanning /mnt/btrfs-vol/rootfs/btrbk-snap, generation is still 62651
Scanning volume / generations from 63842 to 63842, with size cutoff 1
00.09 Scanned 33132 retained 1
Deduplicating filesystem <home>
File 'ravi/.config/google-chrome/Default/Local Storage/chrome-extension_loljledaigphbcpfhfmgopdkppkifgno_0.localstorage' is in use, skipping
02.39 Size group 10/3844 (3201024) sampled 20 hashed 6 freed 0
You need to run this command as root.

[HaleTom]

It seems the workaround is to run su root first.

Why doesn't sudo work? Is there a work around for having sudo work to allow this to be run non-interactively?

[rodrigoscoelho]

I'm with the same problem.

[rodrigoscoelho]

My problem was an immutable Attr. I did a "lsattr -R | grep "i--", found the file with immutable attribute, did a chattr -i "file". DONE.

[MarcelWaldvogel]

I just ran into the same problem.

1. Immutable should not prevent the deduplication, as it does neither modify the (user-visible) contents of the file nor allow later modification that immutable would prevent. This is currently required by opening with O_RDWR. Is this a precondition enforced by the kernel?
2. The error message as root should be different.
   • EPERM as root should say "Permission denied" (which should included for non-root users as well)
   • At least for root users, the offending file should be named.
3. When running as root or EPERM does not apply to the first file, bedup should continue instead of aborting.

[MarcelWaldvogel]

Looking at the code changed in #97, it seems that earlier, EACCESS was returned on immutable (at least that's how it is handled). Now, special handling for EPERM is added. Feel free to try.

BTW: Do we have to assume that @g2p is inactive?

[DavidGoodwin]

this fork might contain a fix - https://github.com/galvanized/bedup

[MarcelWaldvogel]

#97 works fine for me

[henfri]

Hello,

I am confused. I am running bedup from pip. How can I fix this? @MarcelWaldvogel Do you say that #97 fixes this for you, or do you say that #97 did not introduce the problem?

Greetings, Hendrik

[MarcelWaldvogel]

#97 does not fix the cause, but makes the error message more helpful (mentions immutable) and improves the behavior (does not abort when it probably is only due to immutable).

I.e., it logs the real problem (at least in my case) and continues with the dedup, so that the problem can be fixed at the operator's convenience.

To fix it, you can apply [https://github.com/g2p/bedup/pull/97/commits/80fb85200ce32107fc7015d1f56f79e6ddf237de](the patch) manually (or with the patch command). To find out the install location pip used, check the output of pip list -v.

[MarcelWaldvogel]

BTW: I have since stopped using deduplication on btrfs. Deduplication was very successful on my backup disks (created using rsync --link-dest), probably too successful. It resulted in slowing down many operations; especially mounting often took more than a minute and removing subtrees with a lot of successful dedups were extremely slow.

In my case, I switched to hardlink, which does not have these side effects (probably stemming from the additional dedup metadata and having to update it).

[lilydjwg]

This has occurred to me yesterday. chattr -i the path mentioned twice fixes it (for once). It seems that somehow bedup left those attributes last time.

[MarcelWaldvogel]

@lilydjwg: My cause of immutable attributes was terminating a bedup dedup run with ^C

[cobrafast]

I just upgraded to Linux 5.1 and now bedup stopped working for me as well. However, in my case there are no immutable files. I used su - to become root, so it doesn't get rooter than this.

# uname -r
5.1.2-arch1-1-lost-LOST

# id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),19(log)

# lsattr -R /mnt/8t3 | grep 'i--' | wc -l
0

# bedup dedup /mnt/8t3
Skipped 1 frozen volumes in filesystem <8t3>
Scanning volume /mnt/8t3 generations from 2186 to 2186, with size cutoff 8388608
00.00 Scanned  retained 0
Deduplicating filesystem <8t3>
00.44 Size group 4/7 (9190813) sampled 8 hashed 4 freed 0
You need to run this command as root.

What else can I try?

[cobrafast]

I tried to debug the problem but didn't get very far. All I found out is that it actually is the fcntl.ioctl() call for BTRFS_IOC_CLONE that's throwing "Operation not permitted".

Traceback (most recent call last):                                                                                                                    
  File "/root/.local/lib/python3.7/site-packages/bedup-0.10.1-py3.7-linux-x86_64.egg/bedup/__main__.py", line 487, in main                            
    return args.action(args)                                                                                                                          
  File "/root/.local/lib/python3.7/site-packages/bedup-0.10.1-py3.7-linux-x86_64.egg/bedup/__main__.py", line 197, in vol_cmd                         
    dedup_tracked(sess, volset, tt, defrag=args.defrag)                                                                                               
  File "/root/.local/lib/python3.7/site-packages/bedup-0.10.1-py3.7-linux-x86_64.egg/bedup/tracking.py", line 405, in dedup_tracked                   
    dedup_tracked1(ds, comm1)                                                                                                                         
  File "/root/.local/lib/python3.7/site-packages/bedup-0.10.1-py3.7-linux-x86_64.egg/bedup/tracking.py", line 609, in dedup_tracked1                  
    dedup_fileset(ds, fileset, fd_names, fd_inodes, size)                                                                                             
  File "/root/.local/lib/python3.7/site-packages/bedup-0.10.1-py3.7-linux-x86_64.egg/bedup/tracking.py", line 632, in dedup_fileset                   
    deduped = clone_data(dest=dfd, src=sfd, check_first=True)                                                                                         
  File "/root/.local/lib/python3.7/site-packages/bedup-0.10.1-py3.7-linux-x86_64.egg/bedup/platform/btrfs.py", line 602, in clone_data                
    ioctl_pybug(dest, lib.BTRFS_IOC_CLONE, src)                                                                                                       
  File "/root/.local/lib/python3.7/site-packages/bedup-0.10.1-py3.7-linux-x86_64.egg/bedup/platform/btrfs.py", line 361, in ioctl_pybug               
    return fcntl.ioctl(fd, ioc, arg)                                                                                                                  
PermissionError: [Errno 1] Operation not permitted

However, when I use cp -av --reflink=always to dedupe the offenders (which is dodgy as it might screw up permissions, attributes, and dates), no error is thrown and the files are successfully deduplicated (at least they produce the same hash with fienode, meaning their data is stored in the same physical extents).

[trenner02]

@cobrafast Have you found a solution to this? I've just run into the exact same problem. Of the two files it's attempting to dedup, one of them was immutable and I hit the appropriate error from this patch: https://github.com/g2p/bedup/pull/97 . Once I removed the immutable attribute I'm hitting that exact same stack trace you have.

Kernel: 5.0.0-37-generic on Ubuntu 18.04.3 LTS.