borgbackup
bad / corrupted key contents
borg 1.2.4 gives a not very helpful error msg if there is crap in the borg key:
exlumine@debian:/mnt/backup/x230$ borg info /mnt/backup/x230/
Local Exception
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/borg/archiver.py", line 5213, in main
exit_code = archiver.run(args)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/borg/archiver.py", line 5144, in run
return set_ec(func(args))
^^^^^^^^^^
File "/usr/lib/python3/dist-packages/borg/archiver.py", line 170, in wrapper
kwargs['manifest'], kwargs['key'] = Manifest.load(repository, compatibility)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/borg/helpers/manifest.py", line 187, in load
key = key_factory(repository, cdata)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/borg/crypto/key.py", line 134, in key_factory
return identify_key(manifest_data).detect(repository, manifest_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/borg/crypto/key.py", line 590, in detect
if not key.load(target, passphrase):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/borg/crypto/key.py", line 767, in load
success = self._load(key_data, passphrase)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/borg/crypto/key.py", line 611, in _load
cdata = a2b_base64(key_data)
^^^^^^^^^^^^^^^^^^^^
ValueError: string argument should contain only ASCII characters
Platform: Linux debian 6.1.0-17-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.69-1 (2023-12-30) x86_64
Linux: Unknown Linux
Borg: 1.2.4 Python: CPython 3.11.2 msgpack: 1.0.3 fuse: pyfuse3 3.2.1 [pyfuse3,llfuse]
PID: 15931 CWD: /mnt/backup/x230
sys.argv: ['/usr/bin/borg', 'info', '/mnt/backup/x230/']
SSH_ORIGINAL_COMMAND: None
Desired behaviour: "Your borg key is corrupted: {path_of_key_or_url_of_repo}"
Hey I am relatively new to this organization and new to open source as well . I find that no one is assigned to this issue , would you please guide me on how can i fix this issue and start contributing towards it ?
@SayeedKhan21 if issue is not assigned to anyone you can take it. Just make a pull request
is there any contributing guidelines or any thing as such ? coz it would be easy for me to then follow that and make a PR maybe .I dont have any idea of how should i do the setup on my system and from where should i start working on this issue
hey @infectormp do you'all have a discord server or any platform as such where i could ask my doubts ? Or is it okay for me to ask my doubts here ?
hey @infectormp , i have done a basic setup on my system following the guidelines , could u direct me more towards what should be done next ? Like how should i proceed towards contributing to the issue and if possible could u provide more insights on the issue mentioned
I'm not familiar with the development process, so hopefully @ThomasWaldmann can help you out.
@SayeedKhan21 you need a git/source based setup (see our docs).
clone the borgbackup/borg repo to SayeedKahn21/borg, clone that to your local machine. checkout 1.2-maint branch, then git checkout -b fix-8016, so you have your own new branch to work on.
then try to build (from source) and run borg "as is". run the tests (via tox or pytest), all tests should pass.
then reproduce the issue (see top post), fix it, commit and push to your gh repo, make a PR against borgbackup/borg 1.2-maint branch.
@SayeedKhan21 you need to follow the borg install docs, esp. the part that talks about system-level requirements, like libraries and header file.
On Linux you need to install these via the linux package manager. In your case openssl lib or headers are missing, but there are also some others you need.
@ThomasWaldmann i could resolve that error by installing different libraries, however i am getting new errors now
@SayeedKhan21 please rather copy&paste text than uploading images, if possible.
Does git tag list some 10-50 tags in your local repo workdir?
Did you pip install -e . while having your virtual env active?
@ThomasWaldmann I will make sure I copy&paste the error text from next time, sorry for the inconvinience . I ran pip install -e . command and everything runs successfully , however git tag does not list anything . Do I need to checkout to my desired fix-8016 branch before doing all of this ? Running tox after all this still gives same error
@ThomasWaldmann i repeated the entire process following installation docs correctly and this is the error that I am facing
(borg-env) sayeed@dell-g3:~/Sayeed/OPEN SOURCE/borg$ fakeroot -u tox --skip-missing-interpreters
py39-none: skipped because could not find python interpreter with spec(s): py39
py39-none: SKIP ⚠ in 0.02 seconds
py39-fuse2: skipped because could not find python interpreter with spec(s): py39
py39-fuse2: SKIP ⚠ in 0 seconds
py39-fuse3: skipped because could not find python interpreter with spec(s): py39
py39-fuse3: SKIP ⚠ in 0 seconds
py310-none: install_deps> python -I -m pip install -r requirements.d/development.txt
.pkg: install_requires> python -I -m pip install 'Cython>=3' pkgconfig setuptools 'setuptools_scm[toml]>=6.2' wheel
.pkg: _optional_hooks> python '/home/sayeed/Sayeed/OPEN SOURCE/borg-env/lib/python3.10/site-packages/pyproject_api/_backend.py' True setuptools.build_meta
.pkg: get_requires_for_build_sdist> python '/home/sayeed/Sayeed/OPEN SOURCE/borg-env/lib/python3.10/site-packages/pyproject_api/_backend.py' True setuptools.build_meta
py310-none: install_package_deps> python -I -m pip install 'Cython>=3' argon2-cffi 'msgpack<=1.0.7,>=1.0.3' packaging pkgconfig 'platformdirs<5.0.0,>=2.6.0; sys_platform != "darwin"' 'platformdirs<5.0.0,>=3.0.0; sys_platform == "darwin"' setuptools 'setuptools_scm[toml]>=6.2' wheel
py310-none: install_package> python -I -m pip install --force-reinstall --no-deps -e '/home/sayeed/Sayeed/OPEN SOURCE/borg'
py310-none: commands[0]> py.test -v -n 1 -rs --cov=borg --cov-config=.coveragerc --benchmark-skip --pyargs borg.testsuite
============================================================== test session starts ==============================================================
platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.3.0 -- /home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/bin/python
cachedir: .tox/py310-none/.pytest_cache
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/sayeed/Sayeed/OPEN SOURCE/borg
configfile: pyproject.toml
plugins: xdist-3.5.0, cov-4.1.0, benchmark-4.0.0
initialized: 1/1 workerINTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR> config.hook.pytest_sessionstart(session=session)
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR> return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR> raise exc.with_traceback(exc.__traceback__)
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/xdist/dsession.py", line 83, in pytest_sessionstart
INTERNALERROR> nodes = self.nodemanager.setup_nodes(putevent=self.queue.put)
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/xdist/workermanage.py", line 68, in setup_nodes
INTERNALERROR> return [self.setup_node(spec, putevent) for spec in self.specs]
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/xdist/workermanage.py", line 68, in <listcomp>
INTERNALERROR> return [self.setup_node(spec, putevent) for spec in self.specs]
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/xdist/workermanage.py", line 76, in setup_node
INTERNALERROR> node.setup()
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/xdist/workermanage.py", line 272, in setup
INTERNALERROR> basetemp = self.config._tmp_path_factory.getbasetemp()
INTERNALERROR> File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/py310-none/lib/python3.10/site-packages/_pytest/tmpdir.py", line 182, in getbasetemp
INTERNALERROR> raise OSError(
INTERNALERROR> OSError: The temporary directory /tmp/pytest-of-sayeed is not owned by the current user. Fix this and try again.
py310-none: exit 3 (0.57 seconds) /home/sayeed/Sayeed/OPEN SOURCE/borg> py.test -v -n 1 -rs --cov=borg --cov-config=.coveragerc --benchmark-skip --pyargs borg.testsuite pid=23262
py310-none: FAIL ✖ in 1 minute 17 seconds
"The temporary directory /tmp/pytest-of-sayeed is not owned by the current user. Fix this and try again."
So, rm -rf that directory.
To avoid this (guess this is caused by fakeroot / sudo sometimes), you could just invoke the tests without fakeroot / sudo. Most tests run without that also.
@ThomasWaldmann The other tests passed however ig there's an error related to sphinx related tests , but not getting exactly what is it
docs: install_package> python -I -m pip install --force-reinstall --no-deps -e '/home/sayeed/Sayeed/OPEN SOURCE/borg'
docs: commands[0] /home/sayeed/Sayeed/OPEN SOURCE/borg/docs> sphinx-build -n -v -W --keep-going -b html -d /home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/docs/tmp/doctrees . /home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/docs/tmp/html
Traceback (most recent call last):
File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/docs/lib/python3.10/site-packages/sphinx/cmd/build.py", line 293, in build_main
app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
File "/home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/docs/lib/python3.10/site-packages/sphinx/application.py", line 157, in __init__
raise ApplicationError(__('Cannot find source directory (%s)') %
sphinx.errors.ApplicationError: Cannot find source directory (/home/sayeed/Sayeed/OPEN SOURCE/borg/docs/SOURCE/borg/.tox/docs/tmp/doctrees)
Application error:
Cannot find source directory (/home/sayeed/Sayeed/OPEN SOURCE/borg/docs/SOURCE/borg/.tox/docs/tmp/doctrees)
docs: exit 2 (0.24 seconds) /home/sayeed/Sayeed/OPEN SOURCE/borg/docs> sphinx-build -n -v -W --keep-going -b html -d /home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/docs/tmp/doctrees . /home/sayeed/Sayeed/OPEN SOURCE/borg/.tox/docs/tmp/html pid=48776
.pkg: _exit> python '/home/sayeed/Sayeed/OPEN SOURCE/borg-env/lib/python3.10/site-packages/pyproject_api/_backend.py' True setuptools.build_meta
py39-none: SKIP (0.02 seconds)
py39-fuse2: SKIP (0.00 seconds)
py39-fuse3: SKIP (0.00 seconds)
py310-none: FAIL code 1 (899.80=setup[54.03]+cmd[845.76] seconds)
py310-fuse2: OK (921.04=setup[50.11]+cmd[870.92] seconds)
py310-fuse3: OK (1021.38=setup[50.99]+cmd[970.40] seconds)
py311-none: SKIP (0.01 seconds)
py311-fuse2: SKIP (0.00 seconds)
py311-fuse3: SKIP (0.00 seconds)
py312-none: SKIP (0.00 seconds)
py312-fuse2: SKIP (0.00 seconds)
py312-fuse3: SKIP (0.00 seconds)
docs: FAIL code 2 (51.24=setup[51.00]+cmd[0.24] seconds)
evaluation failed :( (2893.58 seconds)
Please checkout the 1.2-maint branch, it looks like you worked on master.
@ThomasWaldmann after i create my local branch and make changes in it , I should be able to run all the tests in the same branch right ? Like I agree that I was now on master branch , however once I make changes , I should run all the tests in my own local branch right ?
About usage of local branches, I just merged a PR into the master docs - that part also applies to 1.2-maint or 1.4-maint:
https://borgbackup.readthedocs.io/en/master/development.html
And yes, you just run the tests using the code in your local feature / fix branch before / after doing changes.
Ok , so if I run tox command in my local branch fix-8016 , all the tests should ideally pass right ?
@ThomasWaldmann I get the same error ,which is mentioned in above comments , when i ran tox command after I did git checkout -b fix-8016
commit 988d8a1c82a6ecf883ac9de9978faa6fab1242fc (HEAD, origin/1.2-maint)
Merge: 3551f03a 9c860baf
Author: Thomas Waldmann <[email protected]>
Date: Sun Jan 7 19:09:53 2024 +0100
Merge pull request #8031 from kmille/1.2-maint
improve docs for borg with-lock, 1.2-maint backport
I can't reproduce your issue, this is the result of my tox run (on macOS, there is only fuse2 support, no fuse3):
py38-none: OK (450.07=setup[47.48]+cmd[402.59] seconds)
py38-fuse2: OK (413.88=setup[22.54]+cmd[391.34] seconds)
py38-fuse3: FAIL code 1 (3.00 seconds)
py39-none: OK (424.02=setup[21.28]+cmd[402.75] seconds)
py39-fuse2: OK (438.37=setup[26.74]+cmd[411.64] seconds)
py39-fuse3: FAIL code 1 (2.10 seconds)
py310-none: OK (412.42=setup[20.74]+cmd[391.68] seconds)
py310-fuse2: OK (472.18=setup[26.15]+cmd[446.03] seconds)
py310-fuse3: FAIL code 1 (2.32 seconds)
py311-none: OK (521.26=setup[25.09]+cmd[496.17] seconds)
py311-fuse2: OK (689.14=setup[33.88]+cmd[655.26] seconds)
py311-fuse3: FAIL code 1 (2.99 seconds)
py312-none: OK (567.40=setup[39.95]+cmd[527.44] seconds)
py312-fuse2: OK (582.38=setup[45.13]+cmd[537.25] seconds)
py312-fuse3: FAIL code 1 (2.60 seconds)
evaluation failed :( (4984.18 seconds)
So maybe try (always first have your virtual env active and cd to the workdir):
python3 setup.py clean clean2
pip install -Ur requirements.d/development.txt
pip install -e .