Pre-backup shell command will not run unless repository is accessible

[ashleycawley]

trafficstars

To Reproduce Steps to reproduce the behaviour:

1. Setup a remote borg repo on a remote LAN which is only accessible via VPN, configure Vorta to use the repo and do the initial seed (works as expected).
2. Setup Pre-backup shell command in Vorta to run a simple command (either echo hello or something more complex like a command which will connect to the VPN connection eg: nmcli con up id MyBackupVPN)
3. Disconnect from VPN and close Vorta
4. Re-open Vorta and try running the backup will result in failure: Pre-backup command returned non-zero exit code.

I have read in other similar issue threads that this may be caused by the Remote Repo URL being inaccessible at the time the "pre-backup shell command" is ran, even though the error message does not state that. It is entirely possible in numerous circumstances that users may want to use this pre-boot shell command to do things like:

• Mount a samba network share
• Mount a encrypted partition/container
• Or establish a VPN connection over which to perform the backup
• etc.. which would then subsequently make the Repo URL accessible. For me this was a no-brainer and the first use-case that came to mind in an effort to help mitigate against ransomeware and other such attacks.

It is my opinion the pre-backup shell command should be triggered regardless of whether the repo URL is accessible or not at that moment, it should after all be initiated "pre-backup" as advertised.

Thank you for the time and help that everyone has put into making this a great piece of software!

Environment (please complete the following information):

• OS: Zorin OS
• Vorta version: 1.2.0
• Installed from: Built-in Zorin Software Store

[real-yfprojects]

Could it be that no internet resource is accessible right after boot which is the reason for your issue while vorta does not even check whether the borg repo is accessible?

[ashleycawley]

Thanks for your input but I don't believe it is a internet connectivity issue or due to the system having only recently booted because I can reproduce the error with ease on a system that has been online for days and has a continuous reliable ping going to 8.8.8.8

I read somewhere on one of the closed issues it was caused due to the repo URL not being available at the time it executes, which I know my borg repo URL is not available at the moment, because I'm relying on the pre-backup shell command to establish a VPN connection and then the borg repo URL is accessible.

[real-yfprojects]

I looked into the code and vorta will not check whether a repo is accessible before running the pre-backup command. The error message you see can only occur in case the pre-backup command is actually run and return a non-zero return code. In your case this is exactly the problem: Your backup command is not returning the exit code 0. You can check that in bash by running the following:

> cmd # run pre-backup command
> echo $?
0  # in your case a value greater than zero

https://github.com/borgbase/vorta/blob/fd88d7ff2119fca0cfb1891b05494dae735f7399/src/vorta/borg/create.py#L109-L112

[ashleycawley]

OK, this is interesting. But why does a simple command like: echo hello produce the same error? Just for context I'm very familiar with exit status codes and have written elaborate bash scripts before for personal and work.

If that message isn't tied to the success / availability of the repo URL then why does it fail with the same message when you just enter in echo hello? That does exit with exit status code of 0?

Before posting I did also try to take a look at the code and came across the same snippet of code that you highlighted, but I didn't know if there was something more to it elsewhere. I will try again soon with a different simple command.

[real-yfprojects]

Now we are getting close to the issue. On my machine echo hello works fine as a pre-backup command.

[real-yfprojects]

What is the output of the following?

> python3 -c "import subprocess; p = subprocess.run('echo hello', shell=True); print(p)"

[ashleycawley]

hello
CompletedProcess(args='echo hello', returncode=0)

Apologies, I think I may have unknowingly given you false information, I think when I encounter the error, if I then replace the command with echo hello and try to re-run the status / error was not changing. I needed to close down and re-open Vorta to get a true error.

After re-opening Vorta and trying echo hello in the Pre-backup shell command immediately, then it will hang, as expected as it cannot connect to the remote repo (as VPN is disconnected).

So apologies, the echo hello was not a fair test.

I have narrowed it down to the two VPN commands it does not like: Pre-backup (connect to VPN): nmcli con up id Remote Post-backup (disconnect from VPN): nmcli con down id Remote

I'm not sure why it wouldn't like either of those two commands because look:

$ python3 -c "import subprocess; p = subprocess.run('nmcli con up id Remote', shell=True); print(p)"
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/19)
CompletedProcess(args='nmcli con up id Remote', returncode=0)

And...

$ python3 -c "import subprocess; p = subprocess.run('nmcli con down id Remote', shell=True); print(p)"
Connection 'Remote' successfully deactivated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/18)
CompletedProcess(args='nmcli con down id Remote', returncode=0)

I have also tried sending the output of those commands to &>/dev/null in case it was the output/text it wasn't liking, but still the same result.

[ashleycawley]

OK, so I tweaked the code locally, line 110 of create.py to try and circumvent and test this, I reversed the logic for a moment: if cls.pre_post_backup_cmd(ret) == 0: The backup now starts but does not work because it fails to start the VPN connection, even though the same command on the shell will connect to the VPN fine.

I think I'm determining that perhaps this Flatpak(?) installation of it may not have sufficient permissions to initiate or work with the VPN connection? I also tested this further by starting the VPN connection myself, start the backup off (which works now) in order to see if the Post-backup command to take it down/offline would work, that also fails / does not disconnect from the VPN as expected.

So the nmcli con up id Remote and nmcli con down id Remote commands are not working in Vorta but work on my terminal/shells.

I would be thankful of any help or input, happy to do further testing. Thank you for your time and help so far.

[real-yfprojects]

Vorta doesn't capture the output of the pre/post-backup commands. I would therefore try to pipe the output/error stream of nmcli into a file and have a look at the error it reports.

[ashleycawley]

Trying to redirect the output of the command to a file yields no results: nmcli con up id Remote > /tmp/vpn-up.log

Any further suggestions on why Vorta doesn't seem to be running or liking these commands? As mentioned they work in the terminal ran by my user.

[real-yfprojects]

nmcli con up id Remote > /tmp/vpn-up.log

This only redirects the standard output stream. The error message will probably be found in the standard error stream.

[ashleycawley]

This only redirects the standard output stream. The error message will probably be found in the standard error stream.

Well, thanks for pointing out my error in redirecting output, but is that where your help stops? To be helpful you could have elaborated by mentioning just one character to redirect both (&).

It was fruitless anyway: nmcli con up id Remote &>/tmp/vpn-up.log Still yielded no results.

It looks like my earlier guess was right I think:

I think I'm determining that perhaps this Flatpak(?) installation of it may not have sufficient permissions to initiate or work with the VPN connection?

I had a hunch, uninstalled Vorta (which was installed by the Zorin OS Software store / flatpack I think), instead I went with a fresh install using: pip3 install vorta

Imported in my profile/json files and boom, immediately my pre-backup and post-backup VPN commands work perfectly, ran as a standard user as well, no sudo or root.

I implore that people with more expertise than I in the Vorta community figure out why the pre and post backup commands do not work reliably for the flatpack version, it was almost a deal-breaker for me, I was just about to move to other backup software.

[real-yfprojects]

Well, thanks for pointing out my error in redirecting output, but is that where your help stops? To be helpful you could have elaborated by mentioning just one character to redirect both (&).

I forgot that this possibility also exists.

It was fruitless anyway: nmcli con up id Remote &>/tmp/vpn-up.log Still yielded no results.

Software including nmcli should show an error message if it failed its job due any problem including missing permissions.

I had a hunch, uninstalled Vorta (which was installed by the Zorin OS Software store / flatpack I think), instead I went with a fresh install using: pip3 install vorta

Imported in my profile/json files and boom, immediately my pre-backup and post-backup VPN commands work perfectly, ran as a standard user as well, no sudo or root.

That's great!

it was almost a deal-breaker for me, I was just about to move to other backup software.

I am sorry to hear that. Maybe I should have pointed you to the pip distribution earlier but I was more focussed on debugging.

[ashleycawley]

Thanks for your time and help none the less 👍

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.