do_known_hosts: hostkeys_foreach failed: No such file or directory

[KvL159]

I tried to setup Borg following the instructions provided, however it keeps failing. Setup: Yunohost 4.0.8 Buster, tested on 2 servers (same results) : 1 upgraded from Yunohost 3.X and 1 fresh installed 4.X

At first, the install ran into some issues, the package Cython (pip) was missing, I was able to solve this issue by installing it manually, however would it be possible to fix it in the ynh package ?

Here are the details regarding Borg not working using the following command: $ sudo yunohost app install borgserver

Here is the log from journal :

-- The unit home-yunohost.backup-tmp-auto_rainloop-apps-rainloop-backup-var-www-rainloop.mount has successfully entered the 'dead' state.
DATE server.to.backup backup-with-borg[001]: Custom backup method could not get past the 'backup' step
DATE server.to.backup backup-with-borg[001]: Collecting files to be backed up for wordpress...
DATE server.to.backup backup-with-borg[001]: [++++................] > Loading installation settings...
DATE server.to.backup backup-with-borg[001]: [####++++............] > Backing up the main app directory...
DATE server.to.backup backup-with-borg[001]: [########++..........] > Backing up nginx web server configuration...
DATE server.to.backup backup-with-borg[001]: [##########++........] > Backing up php-fpm configuration...
DATE server.to.backup backup-with-borg[001]: [############++++....] > Backing up the MySQL database...
DATE server.to.backup backup-with-borg[001]: [################++..] > Backing up fail2ban configuration...
DATE server.to.backup backup-with-borg[001]: Creating a backup archive from the collected files...
DATE server.to.backup backup-with-borg[001]: do_known_hosts: hostkeys_foreach failed: No such file or directory
DATE server.to.backup backup-with-borg[001]: do_known_hosts: hostkeys_foreach failed: No such file or directory
DATE server.to.backup backup-with-borg[001]: Could not run script: /etc/yunohost/hooks.d/backup_method/05-borg_app
DATE server.to.backup systemd[002]: home-yunohost.backup-tmp-auto_wordpress-apps-wordpress-backup-var-www-wordpress.mount: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- The unit UNIT has successfully entered the 'dead' state.
DATE server.to.backup systemd[1]: home-yunohost.backup-tmp-auto_wordpress-apps-wordpress-backup-var-www-wordpress.mount: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- The unit home-yunohost.backup-tmp-auto_wordpress-apps-wordpress-backup-var-www-wordpress.mount has successfully entered the 'dead' state.
DATE server.to.backup backup-with-borg[001]: Custom backup method could not get past the 'backup' step
DATE server.to.backup systemd[1]: borg.service: Main process exited, code=exited, status=1/FAILURE
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- An ExecStart= process belonging to unit borg.service has exited.
--
-- The process' exit code is 'exited' and its exit status is 1.
DATE server.to.backup systemd[1]: borg.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- The unit borg.service has entered the 'failed' state with result 'exit-code'.
DATE server.to.backup systemd[1]: Failed to start Run backup borg.
-- Subject: A start job for unit borg.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- A start job for unit borg.service has finished with a failure.

The service status :

● borg.service - Run backup borg
   Loaded: loaded (/etc/systemd/system/borg.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since DATE UTC; 3min 7s ago
  Process: 001 ExecStart=/usr/local/bin/backup-with-borg (code=exited, status=1/FAILURE)
 Main PID: 001 (code=exited, status=1/FAILURE)

DATE server.to.backup backup-with-borg[001]: [############++++....] > Backing up the MySQL database...
DATE server.to.backup backup-with-borg[001]: [################++..] > Backing up fail2ban configuration...
DATE server.to.backup backup-with-borg[001]: Creating a backup archive from the collected files...
DATE server.to.backup backup-with-borg[001]: do_known_hosts: hostkeys_foreach failed: No such file or directory
DATE server.to.backup backup-with-borg[001]: do_known_hosts: hostkeys_foreach failed: No such file or directory
DATE server.to.backup backup-with-borg[001]: Could not run script: /etc/yunohost/hooks.d/backup_method/05-borg_app
DATE server.to.backup backup-with-borg[001]: Custom backup method could not get past the 'backup' step
DATE server.to.backup systemd[1]: borg.service: Main process exited, code=exited, status=1/FAILURE
DATE server.to.backup systemd[1]: borg.service: Failed with result 'exit-code'.
DATE server.to.backup systemd[1]: Failed to start Run backup borg.

And the error log from borg :

./05-borg_app: line 31: borg: command not found
./05-borg_app: line 44: borg: command not found

It seems that the borg command isn't working as well, any idea where could be the issue ?

[KvL159]

Here is some update on the issue :

$ borg
-bash: borg: command not found

I then tried to run the script manually : $ cd /usr/local/bin

$ backup-with-borg
ls: cannot access '/etc/yunohost/hooks.d/backup/': No such file or directory
Info: Creating a backup archive from the collected files...
Warning: do_known_hosts: hostkeys_foreach failed: No such file or directory
Warning: do_known_hosts: hostkeys_foreach failed: No such file or directory
Error: Could not run script: /etc/yunohost/hooks.d/backup_method/05-borg_app
Error: Custom backup method could not get past the 'backup' step

It seems related to this issue, however I'm not sure if it was solved : https://forum.yunohost.org/t/borg-backup-sauvegarde-locale/12496/3

@zamentur Thank you for your help

[KvL159]

I am still not able to run Borg on any of my servers, any hint where could be the issue ?

[Thatoo]

I get the same error as you do

nov. 04 22:39:14 xx backup-with-borg[25707]: Échec de l’exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app nov. 04 22:39:14 xx backup-with-borg[25707]: Échec de la méthode de sauvegarde personnalisée à l’étape 'backup' nov. 04 22:39:14 xx systemd[1]: borg.service: Main process exited, code=exited, status=1/FAILURE nov. 04 22:39:14 xx systemd[1]: borg.service: Failed with result 'exit-code'. nov. 04 22:39:14 x systemd[1]: Failed to start Run backup borg.

It seems like borg is not very easy to handle...

[BCITMike]

borg init: error: argument REPOSITORY: Invalid location format: "ssh://[email protected]:repo/~/backup"

There's at least two problems. First, SSH connection was failing, even though the script points to /root/.ssh/id_borg_ed25519. Got around that by adding to /root/.ssh/config:

      Host *.repo.borgbase.com
     IdentityFile ~/.ssh/id_borg_ed25519

That could just be from testing from command line and not from script and missing the variables, though. Won't hurt to add.

Second, I was getting borg init errors, and needed to remove the "ssh://" at the start of the repo. Then I was getting this error: [2020-12-01_03:36:24] Repository path not allowed: /srv/repos/ok5v04j1/repo/~/backup

borg list [email protected]:repo prints the files. borg list [email protected]:repo/~/backup gives: Repository path not allowed: /srv/repos/ok5v04j1/repo/~/backup

So from my notes, I seem to have removed the ~/backup from the repo path as well, and now when I run the backup, the error log contains:

[2020-12-01_03:44:52] A repository already exists at [email protected]:repo.

IIRC, this always happened and can be ignored. But for the first time since upgrading to Debian 10, borgbase shows an updated modified time, so the repo is currently getting written to.

Change: borg_ynh/conf/backup_method.j2 (in repo) or 05-borg_app (on your server) repo=ssh://{{ ssh_user }}@{{ server }}/~/backup to: repo={{ ssh_user }}@{{ server }}

[BCITMike]

@KvL159 @Thatoo ls: cannot access '/etc/yunohost/hooks.d/backup/': No such file or directory

Fix this with: mkdir -p /etc/yunohost/hooks.d/backup

Then, to fix backup, do you recall if you previously edited the repo to remove the "ssh://" and the "/~/backup"?

See my comment above.