backintime icon indicating copy to clipboard operation
backintime copied to clipboard

Published 20 hours ago • verified publisher icon bit-team

feat: Add local gocryptfs support

Open daviewales opened this issue 1 year ago • 50 comments

Blocked by

  • #1865

This is a rebase of Germar Reitze's gocrypt branch on the latest dev branch.

It seems to work, so I thought we should try to merge it as a base for future gocryptfs work.

I've tested backing up a test file, deleting it, then restoring it.

Related to #1734.

daviewales avatar Oct 08 '24 03:10 daviewales

Wow that is a big step. Awesome!

Do we need the latest GoCryptFS when running TavisCI? Are there known Issues with gocryptfs from the Ubuntu repos?

buhtz avatar Oct 08 '24 08:10 buhtz

I suspect that we don't need the latest version. I tested with the version in the latest Linux Mint, and it seemed to work.

daviewales avatar Oct 08 '24 08:10 daviewales

The code fetching the latest version was already in Germar's original branch. I haven't made many changes here, other than fixing merge conflicts.

daviewales avatar Oct 08 '24 08:10 daviewales

Please let me know if you prefer me to solve this merge conflicts.

buhtz avatar Oct 11 '24 19:10 buhtz

I'll do another rebase, and switch Travis to fetch the packaged version of gocryptfs rather than the latest release while I'm at it.

daviewales avatar Oct 15 '24 03:10 daviewales

I've rebased on the latest dev. I also added a quick patch to the qt/configure script to include the new qt/manageprofiles directory when installing.

I confirmed that I can still backup a file, delete it, then restore it.

daviewales avatar Oct 15 '24 05:10 daviewales

When my refactoring work on the manage profiles dialog is finished I think about a release candidate and release 1.5.3. After this we can merge your PR, after intense testing of course, and later release it as 1.6.0 somewhere around Christmas or New Year.

buhtz avatar Oct 15 '24 08:10 buhtz

FYI: I do plan a 1.5.3 release in the near future. After that release I would target your PR for a 1.6.0 release.

buhtz avatar Oct 15 '24 08:10 buhtz

I'll apply the suggestions above when I get a chance. There are also a few lint errors, so I'll fix those up too.

daviewales avatar Oct 16 '24 11:10 daviewales

The remaining lint errors are because there is similarity between the encfs and gocryptfs modules... Perhaps this can be ignored?

daviewales avatar Oct 18 '24 10:10 daviewales

The remaining lint errors are because there is similarity between the encfs and gocryptfs modules... Perhaps this can be ignored?

Usually I would say "no" and recommend to create a base class to avoid code duplication. But we will remove encfs, so I see no problem to add an ignore clause for pylint.

# pylint: disable=duplicate-code

Not sure if this need to be place in both files.

buhtz avatar Oct 18 '24 10:10 buhtz

Note to self: I need to test setting up a new gocryptfs backup from scratch. I've been tested this branch with one which is already configured.

EDIT: I've tested setting up a new backup profile from scratch, and the dialog needs some tweaking as it's missing a few inputs. I'll update it to match the EncFS local dialog.

daviewales avatar Oct 18 '24 11:10 daviewales

Hello David, your PR is the next. I leave it up to you to update your branch to the latest dev version. Then I will do some manual testing in a VM.

Christian

buhtz avatar Nov 13 '24 09:11 buhtz

Thanks Christian, I'll try to find some time in the next week to get this ready. If you have a chance, you might be able to help me figure out why the new GoCryptFS profile creation screen is missing a few important fields compared to the old EncFS local screen. Otherwise, I'll see if I can figure it out.

daviewales avatar Nov 14 '24 07:11 daviewales

why the new GoCryptFS profile creation screen is missing a few important fields compared to the old EncFS local screen. Otherwise, I'll see if I can figure it out.

Not sure. Can you show me a screenshot? I am not able to use your current branch because of merge conflicts. I assume they are caused because of my refactoring of the manage profiles dialog.

buhtz avatar Nov 14 '24 14:11 buhtz

Hello David, I will try this weekend to take a closer look to your PR. Don't worry if it isn't finished yet. But before please take care that the PR is up-2-date with your local branch and the latest dev version.

Regards, Christian

buhtz avatar Nov 22 '24 09:11 buhtz

EDIT: I've fixed this in e8a66fa485a97b4643cd5dc1eebb67bdbed6c2ac.

When I create a new EncFS local profile, I see these fields:

image

But when I create a new gocryptfs local profile, I see these fields:

image

i.e. the Where to save snapshots field is missing.

Do you know how to make it appear?

daviewales avatar Nov 29 '24 23:11 daviewales

Also, I've rebased on the latest dev.

daviewales avatar Nov 29 '24 23:11 daviewales

I'm getting the following error when I click 'OK' to finalise creation of a new profile:

DEBUG: [common/config.py:320 Config.checkConfig] Check profile gocryptfs-local
Traceback (most recent call last):
  File "/usr/share/backintime/qt/manageprofiles/__init__.py", line 794, in accept
    if self.validate():
       ^^^^^^^^^^^^^^^
  File "/usr/share/backintime/qt/manageprofiles/__init__.py", line 553, in validate
    if not self.config.setupCron():
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/backintime/common/config.py", line 1468, in setupCron
    self.setupUdev.clean()
  File "/usr/share/backintime/common/tools.py", line 2441, in clean
    self.iface.clean()
  File "/usr/lib/python3/dist-packages/dbus/proxies.py", line 141, in __call__
    return self._connection.call_blocking(self._named_service,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/dbus/connection.py", line 634, in call_blocking
    reply_message = self.send_message_with_reply_and_block(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.AccessDenied: Rejected send message, 1 matched rules; type="method_call", sender=":1.201" (uid=1000 pid=100353 comm="/usr/bin/python3 -Es /usr/share/backintime/qt/app." label="unconfined") interface="net.launchpad.backintime.serviceHelper.UdevRules" member="clean" error name="(unset)" requested_reply="0" destination=":1.88" (uid=0 pid=16206 comm="/usr/bin/python3 -Es /usr/share/backintime/qt/serv" label="unconfined")

Do you have any thoughts?

daviewales avatar Nov 29 '24 23:11 daviewales

Mhm... I am not able to reproduce your dbus related error. But I do see something like this. image

So I executed the gocryptfs command from the error message directly in the shell:

$ gocryptfs -extpass backintime-askpass -quiet /home/user/Dokumente /home/user/.local/share/backintime/mnt/1CD92332/mountpoint
WARNING: Failed to connect to Udev serviceHelper daemon via D-Bus: org.freedesktop.DBus.Error.Spawn.ChildExited
WARNING: D-Bus message: Launch helper exited with unknown return code 1
WARNING: Udev-based profiles cannot be changed or checked due to Udev serviceHelper connection failure
extpass: password is empty

"password is empty" is the relevant message.

EDIT: Currently I have no idea.

EDIT2: I am assuming for someone who understands DBus your error message would be easy to understand. But for me. ;) What kind of system do you use?

AccessDenied: Rejected send message

?

interface="net.launchpad.backintime.serviceHelper.UdevRules"

Did you tried to setup udev scheduling?

comm="/usr/bin/python3 -Es /usr/share/backintime/qt/serv" label="unconfined")

That is interesting. The file /usr/share/backintime/qt/serv does not exist on my system.

EDIT3: I am assuming we do have two separate problems here. On your system something doesn't work related to DBus. On my system something is wrong with backintime-askpass. Further I am assuming that my problem will occur on your system too if you fix your DBus problem.

buhtz avatar Nov 30 '24 09:11 buhtz

I have another gocryptfs profile which I created in another experimental branch. That was working before my latest rebase. Now it's stopped working with a similar extpass: password is empty message. So something strange is happening with backintime-askpass.

daviewales avatar Nov 30 '24 22:11 daviewales

That is a trail. If you can identify a commit where it worked, we can do a diff.

buhtz avatar Dec 01 '24 07:12 buhtz

I can also confirm that the DBusException above occurs even on the latest dev, with a completely new local profile.

I've run git bisect, and for each bisection I've deleted the config, and created a new local profile from scratch. This appears to be the first commit with the issue, but I have no idea why! 59746799b31a853b071f22fe67501e315d3e2633

daviewales avatar Dec 03 '24 03:12 daviewales

Another experiment, git revert 59746799b31a853b071f22fe67501e315d3e2633 on this branch makes everything work!

image

daviewales avatar Dec 03 '24 03:12 daviewales

It looks like adding the license information comment to net.launchpad.backintime.serviceHelper.conf broke DBus!

daviewales avatar Dec 03 '24 03:12 daviewales

I've sent a PR to fix this: #1958

daviewales avatar Dec 03 '24 05:12 daviewales

I still get this askpass error I described. Can you reproduce it?

buhtz avatar Dec 07 '24 16:12 buhtz

I'm not seeing any errors on this branch now. Could you try running with a fresh config file? How exactly are you running it?

daviewales avatar Dec 07 '24 21:12 daviewales

What does the output from --debug look like just before you get that error? Which secret storage backend does the log indicate you are using?

daviewales avatar Dec 07 '24 21:12 daviewales

I do run this on a VM with Debian 12. This is the full debug output.

user@worf ~/backtinime_daviewales/qt (gocrypt-rebase)$ ./backintime-qt --debug
DEBUG: [common/backintime.py:616 argParse] Argument(s) used: {'debug': True, 'quiet': False}
DEBUG: [common/backintime.py:515 startApp] backintime: {'name': 'Back In Time', 'version': '1.6.0-dev.506b31c6', 'running-as-root': False}; host-setup: {'OS': {'/etc/os-release': 'Debian GNU/Linux 12 (bookworm)', '/etc/debian_version': '12.8\n'}}

Back In Time
Version: 1.6.0-dev.506b31c6

Back In Time comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; type `backintime --license' for details.

DEBUG: [common/configfile.py:591 Config.setCurrentProfile] Change current profile: 1=Hauptprofil
WARNING: [common/tools.py:2382 __init__] Failed to connect to Udev serviceHelper daemon via D-Bus: org.freedesktop.DBus.Error.Spawn.ChildExited
WARNING: [common/tools.py:2384 __init__] D-Bus message: Launch helper exited with unknown return code 1
WARNING: [common/tools.py:2385 __init__] Udev-based profiles cannot be changed or checked due to Udev serviceHelper connection failure
DEBUG: [common/tools.py:226 initiate_translation] No language code. Use systems current locale.
DEBUG: [common/tools.py:241 initiate_translation] Language code used: "de"
DEBUG: [common/backintime.py:698 getConfig] config file: "/home/user/.config/backintime/config"; share path: "/home/user/.local/share/backintime"; profiles: "1=Hauptprofil"
DEBUG: [common/pluginmanager.py:233 PluginManager.load] Register plugin path /home/user/backtinime_daviewales/common/plugins
DEBUG: [common/pluginmanager.py:233 PluginManager.load] Register plugin path /home/user/backtinime_daviewales/qt/plugins
DEBUG: [common/tools.py:1084 is_Qt_working] Qt probing result: exit code 2
DEBUG: [common/tools.py:1087 is_Qt_working] Qt probing stdout:

DEBUG: [common/tools.py:1088 is_Qt_working] Qt probing errout:
DEBUG: [common/qt_probing.py:89 <module>] /home/user/backtinime_daviewales/common/qt_probing.py started... Call args: ['/home/user/backtinime_daviewales/common/qt_probing.py', '--debug']
DEBUG: [common/qt_probing.py:90 <module>] Display system: x11
DEBUG: [common/qt_probing.py:91 <module>] XDG_RUNTIME_DIR=/run/user/1000
DEBUG: [common/qt_probing.py:92 <module>] XAUTHORITY=/home/user/.Xauthority
DEBUG: [common/qt_probing.py:93 <module>] QT_QPA_PLATFORM=($QT_QPA_PLATFORM is not set)
DEBUG: [common/qt_probing.py:95 <module>] Current euid: 1000
DEBUG: [common/qt_probing.py:127 <module>] isSystemTrayAvailable for Qt: True
DEBUG: [common/qt_probing.py:132 <module>] /home/user/backtinime_daviewales/common/qt_probing.py is terminating normally (exit code: 2)

DEBUG: [plugins/systrayiconplugin.py:74 init] System tray is available to show the BiT system tray icon
DEBUG: [common/pluginmanager.py:254 PluginManager.load] Add plugin systrayiconplugin.py
DEBUG: [common/pluginmanager.py:254 PluginManager.load] Add plugin notifyplugin.py
DEBUG: [qt/qttools.py:280 createQApplication] QT QPA platform plugin: xcb
DEBUG: [qt/qttools.py:281 createQApplication] QT_QPA_PLATFORMTHEME=<not set>
DEBUG: [qt/qttools.py:286 createQApplication] QT_STYLE_OVERRIDE=<not set>
DEBUG: [qt/qttools.py:289 createQApplication] QT active style: fusion
DEBUG: [qt/qttools.py:290 createQApplication] QT fallback style: 
DEBUG: [qt/qttools.py:291 createQApplication] QT supported styles: ['Windows', 'Fusion']
DEBUG: [qt/qttools.py:292 createQApplication] themeSearchPaths: ['/usr/share/icons', ':/icons']
DEBUG: [qt/qttools.py:293 createQApplication] fallbackSearchPaths: []
DEBUG: [qt/qttools.py:298 createQApplication] Is SystemTray available: True
DEBUG: [qt/qttools.py:320 createQApplication] Trying to set App ID for non-privileged user
DEBUG: [qt/qttools.py:360 initiate_translator] No language code. Use systems current locale.
DEBUG: [qt/icon.py:20 <module>] Checking if the current theme contains the BiT icon...
DEBUG: [qt/icon.py:32 <module>] Found an installed theme: Tango
DEBUG: [common/config.py:1413 Config.isConfigured] Profile (profile_id=None) is not configured because snapshot path is "False" and/or includes are "False".
DEBUG: [manageprofiles/tab_general.py:583 _snapshot_mode_combobox] snapshot_modes={'local': 'Lokal', 'ssh': 'SSH', 'local_encfs': 'Lokal verschlüsselt (EncFS)', 'ssh_encfs': 'SSH verschlüsselt', 'local_gocryptfs': 'Lokal verschlüsselt (gocryptfs)'}
DEBUG: [common/tools.py:1475 keyringSupported] Keyring config file directory: /home/user/.config/python_keyring
DEBUG: [common/tools.py:1490 keyringSupported] Available keyring backends:
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.libsecret.Keyring (priority: 4.8)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.SecretService.Keyring (priority: 5)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.fail.Keyring (priority: 0)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.chainer.ChainerBackend (priority: 10)
DEBUG: [common/tools.py:1550 keyringSupported] Not found Metaclasses: ['keyring.backends.Gnome.Keyring', 'keyring.backends.kwallet.Keyring', 'keyring.backend.SecretServiceKeyring', 'keyring.backend.GnomeKeyring', 'keyring.backend.KDEWallet']
DEBUG: [common/tools.py:1551 keyringSupported] Available supported backends: [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>, <class 'keyring.backends.chainer.ChainerBackend'>]
DEBUG: [common/tools.py:1554 keyringSupported] Found appropriate keyring 'keyring.backends.chainer'
DEBUG: [common/config.py:1413 Config.isConfigured] Profile (profile_id='1') is not configured because snapshot path is "False" and/or includes are "False".
DEBUG: [common/tools.py:1475 keyringSupported] Keyring config file directory: /home/user/.config/python_keyring
DEBUG: [common/tools.py:1490 keyringSupported] Available keyring backends:
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.libsecret.Keyring (priority: 4.8)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.SecretService.Keyring (priority: 5)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.fail.Keyring (priority: 0)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.chainer.ChainerBackend (priority: 10)
DEBUG: [common/tools.py:1550 keyringSupported] Not found Metaclasses: ['keyring.backends.Gnome.Keyring', 'keyring.backends.kwallet.Keyring', 'keyring.backend.SecretServiceKeyring', 'keyring.backend.GnomeKeyring', 'keyring.backend.KDEWallet']
DEBUG: [common/tools.py:1551 keyringSupported] Available supported backends: [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>, <class 'keyring.backends.chainer.ChainerBackend'>]
DEBUG: [common/tools.py:1554 keyringSupported] Found appropriate keyring 'keyring.backends.chainer'
DEBUG: [common/tools.py:1475 keyringSupported] Keyring config file directory: /home/user/.config/python_keyring
DEBUG: [common/tools.py:1490 keyringSupported] Available keyring backends:
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.libsecret.Keyring (priority: 4.8)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.SecretService.Keyring (priority: 5)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.fail.Keyring (priority: 0)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.chainer.ChainerBackend (priority: 10)
DEBUG: [common/tools.py:1550 keyringSupported] Not found Metaclasses: ['keyring.backends.Gnome.Keyring', 'keyring.backends.kwallet.Keyring', 'keyring.backend.SecretServiceKeyring', 'keyring.backend.GnomeKeyring', 'keyring.backend.KDEWallet']
DEBUG: [common/tools.py:1551 keyringSupported] Available supported backends: [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>, <class 'keyring.backends.chainer.ChainerBackend'>]
DEBUG: [common/tools.py:1554 keyringSupported] Found appropriate keyring 'keyring.backends.chainer'
DEBUG: [common/config.py:1413 Config.isConfigured] Profile (profile_id=None) is not configured because snapshot path is "False" and/or includes are "False".
DEBUG: [common/tools.py:1475 keyringSupported] Keyring config file directory: /home/user/.config/python_keyring
DEBUG: [common/tools.py:1490 keyringSupported] Available keyring backends:
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.libsecret.Keyring (priority: 4.8)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.SecretService.Keyring (priority: 5)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.fail.Keyring (priority: 0)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.chainer.ChainerBackend (priority: 10)
DEBUG: [common/tools.py:1550 keyringSupported] Not found Metaclasses: ['keyring.backends.Gnome.Keyring', 'keyring.backends.kwallet.Keyring', 'keyring.backend.SecretServiceKeyring', 'keyring.backend.GnomeKeyring', 'keyring.backend.KDEWallet']
DEBUG: [common/tools.py:1551 keyringSupported] Available supported backends: [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>, <class 'keyring.backends.chainer.ChainerBackend'>]
DEBUG: [common/tools.py:1554 keyringSupported] Found appropriate keyring 'keyring.backends.chainer'
DEBUG: [common/gocryptfstools.py:128 GocryptfsMount.isConfigured] No config in /home/user/backup_dest/gocryptfs.conf
DEBUG: [common/gocryptfstools.py:76 GocryptfsMount.init] Provide password through temp FIFO
DEBUG: [common/gocryptfstools.py:85 GocryptfsMount.init] Call command to create gocryptfs config file: gocryptfs -extpass backintime-askpass -init /home/user/backup_dest

...ERROR DIALOG appears here....

DEBUG: [common/tools.py:1475 keyringSupported] Keyring config file directory: /home/user/.config/python_keyring
DEBUG: [common/tools.py:1490 keyringSupported] Available keyring backends:
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.libsecret.Keyring (priority: 4.8)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.SecretService.Keyring (priority: 5)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.fail.Keyring (priority: 0)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.chainer.ChainerBackend (priority: 10)
DEBUG: [common/tools.py:1550 keyringSupported] Not found Metaclasses: ['keyring.backends.Gnome.Keyring', 'keyring.backends.kwallet.Keyring', 'keyring.backend.SecretServiceKeyring', 'keyring.backend.GnomeKeyring', 'keyring.backend.KDEWallet']
DEBUG: [common/tools.py:1551 keyringSupported] Available supported backends: [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>, <class 'keyring.backends.chainer.ChainerBackend'>]
DEBUG: [common/tools.py:1554 keyringSupported] Found appropriate keyring 'keyring.backends.chainer'
DEBUG: [common/gocryptfstools.py:128 GocryptfsMount.isConfigured] No config in /home/user/backup_dest/gocryptfs.conf
DEBUG: [common/gocryptfstools.py:76 GocryptfsMount.init] Provide password through temp FIFO
DEBUG: [common/gocryptfstools.py:85 GocryptfsMount.init] Call command to create gocryptfs config file: gocryptfs -extpass backintime-askpass -init /home/user/backup_dest
DEBUG: [common/tools.py:1475 keyringSupported] Keyring config file directory: /home/user/.config/python_keyring
DEBUG: [common/tools.py:1490 keyringSupported] Available keyring backends:
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.libsecret.Keyring (priority: 4.8)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.SecretService.Keyring (priority: 5)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.fail.Keyring (priority: 0)
DEBUG: [common/tools.py:1494 keyringSupported] keyring.backends.chainer.ChainerBackend (priority: 10)
DEBUG: [common/tools.py:1550 keyringSupported] Not found Metaclasses: ['keyring.backends.Gnome.Keyring', 'keyring.backends.kwallet.Keyring', 'keyring.backend.SecretServiceKeyring', 'keyring.backend.GnomeKeyring', 'keyring.backend.KDEWallet']
DEBUG: [common/tools.py:1551 keyringSupported] Available supported backends: [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>, <class 'keyring.backends.chainer.ChainerBackend'>]
DEBUG: [common/tools.py:1554 keyringSupported] Found appropriate keyring 'keyring.backends.chainer'
DEBUG: [common/gocryptfstools.py:128 GocryptfsMount.isConfigured] No config in /home/user/backup_dest/gocryptfs.conf
DEBUG: [common/gocryptfstools.py:76 GocryptfsMount.init] Provide password through temp FIFO
DEBUG: [common/gocryptfstools.py:85 GocryptfsMount.init] Call command to create gocryptfs config file: gocryptfs -extpass backintime-askpass -init /home/user/backup_dest
DEBUG: [common/configfile.py:591 Config.setCurrentProfile] Change current profile: 1=Hauptprofil
DEBUG: [common/config.py:1413 Config.isConfigured] Profile (profile_id=None) is not configured because snapshot path is "False" and/or includes are "False".
DEBUG: [common/config.py:1413 Config.isConfigured] Profile (profile_id=None) is not configured because snapshot path is "False" and/or includes are "False".

I am not able to finishe the profile creation because of this error dialog.

image

buhtz avatar Dec 08 '24 12:12 buhtz