borgbackup
utf-8 decode error while answering a yes/no question about changing locations
$ bash convert.sh
Warning: The repository at location ssh://HEL1-AB12/home/~/borg was previously located at ssh://vidar/~/backups/borg
Do you want to continue? [yN] y
Local Exception
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/borg/archiver.py", line 5115, in main
exit_code = archiver.run(args)
File "/usr/lib/python3.10/site-packages/borg/archiver.py", line 5046, in run
return set_ec(func(args))
File "/usr/lib/python3.10/site-packages/borg/archiver.py", line 174, in wrapper
assert_secure(repository, kwargs['manifest'], self.lock_wait)
File "/usr/lib/python3.10/site-packages/borg/cache.py", line 209, in assert_secure
sm.assert_secure(manifest, manifest.key, lock_wait=lock_wait)
File "/usr/lib/python3.10/site-packages/borg/cache.py", line 175, in assert_secure
self._assert_secure(manifest, key, cache_config)
File "/usr/lib/python3.10/site-packages/borg/cache.py", line 181, in _assert_secure
self.assert_location_matches(cache_config)
File "/usr/lib/python3.10/site-packages/borg/cache.py", line 127, in assert_location_matches
if not yes(msg, false_msg="Aborting.", invalid_msg="Invalid answer, aborting.",
File "/usr/lib/python3.10/site-packages/borg/helpers/yes.py", line 80, in yes
answer = input()
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdd in position 150: invalid continuation byte
Platform: Linux odin 5.17.5-arch1-1 #1 SMP PREEMPT Wed, 27 Apr 2022 20:56:11 +0000 x86_64
Linux: Unknown Linux
Borg: 1.2.1 Python: CPython 3.10.5 msgpack: 1.0.3 fuse: None [pyfuse3,llfuse]
PID: 215438 CWD: /home/ruben/backup-to-borg
sys.argv: ['/usr/bin/borg', 'import-tar', '--stats', '--progress', '--upload-buffer', '1500', '--chunker-params=buzhash,10,20,15,4095', '--compression', 'auto,zstd,22', 'HEL1-AB12:/home/~/borg::i3-root-2020-06-10T09:41:40', '--timestamp', '2020-06-10T09:41:40', '--comment', 'import', '-']
SSH_ORIGINAL_COMMAND: None
[ruben@odin backup-to-borg]$ y
-bash: y: command not found
I moved a borg archive from one location to another and got the question about it. I answered with a 'y' and hit enter and after 1-2 seconds I got this backtrace about a Unicode error.
Furthermore, I can reproduce it.
I connect to the server in question via mosh, but the same result is the case when I connect to it via ssh.
I rechecked the archive by running rsync again with rsync -av --checksum source_server:~/borg . and there were no issues with the archive.
Not only that, but I also ran borg check --progress borg on the repository locally and there were no issues reported.
Have you checked borgbackup docs, FAQ, and open Github issues?
Sure
Is this a BUG / ISSUE report or a QUESTION?
BUG
System information. For client/server mode post info for both machines.
Client (Local machine having the issue):
$ neofetch
-` [email protected]
.o+` -----------------------
`ooo/ OS: Arch Linux x86_64
`+oooo: Host: KVM Server RS 4000 G9.5
`+oooooo: Kernel: 5.17.5-arch1-1
-+oooooo+: Uptime: 66 days, 21 hours, 29 mins
`/:-:++oooo+: Packages: 546 (pacman)
`/++++/+++++++: Shell: bash 5.1.16
`/++++++++++++++: Resolution: 1024x768
`/+++ooooooooooooo/` Terminal: /dev/pts/1
./ooosssso++osssssso+` CPU: AMD EPYC 7702P (10) @ 1.996GHz
.oossssso-````/ossssss+` GPU: 00:02.0 Vendor 1234 Device 1111
-osssssso. :ssssssso. Memory: 29201MiB / 32104MiB
:osssssss/ osssso+++.
/ossssssss/ +ssssooo/-
`/ossssso+/:- -:/+osssso+-
`+sso+:-` `.-/+oso:
`++:. `-/+/
.` `/
Filesystem: ZFS
Server (created the repo which was fetched via rsync):
- CentOS 7
- 2 GHz One Core
- 4 GB Memory
- Unknown Filesystem
Your borg version (borg -V).
borg 1.2.1
The borg archive was transferred via rsync from a server which has the following version of borg installed:
borg 1.1.18
How much data is handled by borg?
Around 400 GB
Are you sure that you use English char for y?
please past the locale output here
https://bytetool.web.app/en/ascii/code/0xdd/
Or maybe there was some accent in the buffer and it was combined with the y to form the character @infectormp found.
Are you sure that you use English char for
y?
Well, I pressed 'y' on my keyboard and tried twice (via mosh and ssh). Keyboard is German, locale on my notebook is English. Locale on the server is English too.
Notebook (where I start my journey to the server via mosh/ssh):
$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_US.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=en_US.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=en_US.UTF-8
LC_NAME=en_US.UTF-8
LC_ADDRESS=en_US.UTF-8
LC_TELEPHONE=en_US.UTF-8
LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION=en_US.UTF-8
LC_ALL=
The server:
$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_US.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=en_US.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=en_US.UTF-8
LC_NAME=en_US.UTF-8
LC_ADDRESS=en_US.UTF-8
LC_TELEPHONE=en_US.UTF-8
LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION=en_US.UTF-8
LC_ALL=
If you're wondering. The script is minimal. It just loops through the snapshots stored in restic and imports them one by one:
#!/bin/bash
set -e
export BORG_PASSPHRASE='moppelkotze1'
export RESTIC_PASSWORD='moppelkotze2'
restic snapshots -r i3 | tail -n +4 > snapshots_i3
while IFS= read -r -d $'\n' line; do
_id=$(echo "$line" | cut -d ' ' -f1)
_date=$(echo "$line" | cut -d ' ' -f3)
_time=$(echo "$line" | cut -d ' ' -f4)
_machine=$(echo "$line" | cut -d ' ' -f6)
echo ":: importing snapshot '$_id' from restic from $_date at $_time from machine $_machine"
restic dump --archive tar -r i3 "$_id" "/" | borg import-tar --stats --progress --upload-buffer 1500 --chunker-params=buzhash,10,20,15,4095 --compression auto,zstd,22 "HEL1-BX46:/home/~/backups/borg::${_machine}-root-${_date}T${_time}" --timestamp "${_date}T${_time}" --comment "import from restic" -
done < <(cat snapshots_i3)
That said, I may have entered one of the passwords on the console … maybe those don't get cleared out from the buffer correctly — meaning dead keys doesn't get reset?
@RubenKelevra nice CPU btw. :-)
Ty, but it's just rented ;)
For script use, guess you rather want to set the related BORG_* env var (see the docs for a list), that automatically answers such questions.
Not sure how well input() works if stdin is connected to a pipe from restic.