Segmentation fault while creating an archive

[ayushnix]

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG/ISSUE

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

borg 1.4.0

Operating system (distribution) and version.

Arch Linux, kernel version 6.6.49-1-lts

Hardware / network configuration, and filesystems used.

borg repository was being created on a 16TB Western Digital Ultrastar DC HC550 from a ZFS RAID 10 array locally.

How much data is handled by borg?

Approximately 2.2TB for now.

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg -p create -s -c 300 -C auto,lz4 -e data/videos/_testing /mnt/backup::{now}-{borgversion}-videos /data/videos

Describe the problem you're observing.

Fatal Python error: Segmentation faultata/videos/english-movies/sample-folder-name/sample.mkv                                                                                                

Current thread 0x000076c73cd25b80 (most recent call first):
  File "/usr/lib/python3.12/site-packages/borg/repository.py", line 1713 in write_put
  File "/usr/lib/python3.12/site-packages/borg/repository.py", line 1266 in put
  File "/usr/lib/python3.12/site-packages/borg/cache.py", line 956 in add_chunk
  File "/usr/lib/python3.12/site-packages/borg/archive.py", line 1257 in chunk_processor
  File "/usr/lib/python3.12/site-packages/borg/archive.py", line 1269 in process_file_chunks
  File "/usr/lib/python3.12/site-packages/borg/archive.py", line 1467 in process_file
  File "/usr/lib/python3.12/site-packages/borg/archiver.py", line 699 in _process_any
  File "/usr/lib/python3.12/site-packages/borg/archiver.py", line 797 in _rec_walk
  File "/usr/lib/python3.12/site-packages/borg/archiver.py", line 834 in _rec_walk
  File "/usr/lib/python3.12/site-packages/borg/archiver.py", line 834 in _rec_walk
  File "/usr/lib/python3.12/site-packages/borg/archiver.py", line 834 in _rec_walk
  File "/usr/lib/python3.12/site-packages/borg/archiver.py", line 619 in create_inner
  File "/usr/lib/python3.12/site-packages/borg/archiver.py", line 687 in do_create
  File "/usr/lib/python3.12/site-packages/borg/archiver.py", line 191 in wrapper
  File "/usr/lib/python3.12/site-packages/borg/archiver.py", line 5309 in run
  File "/usr/lib/python3.12/site-packages/borg/archiver.py", line 5391 in main
  File "/usr/sbin/borg", line 8 in <module>

Extension modules: borg.crypto.low_level, msgpack._cmsgpack, borg.chunker, borg.algorithms.checksums, borg.platform.posix, borg.platform.syncfilerange, borg.platform.linux, borg.hashindex, borg.item, borg.compress, llfuse (total: 11)
fish: Job 1, 'borg -p create -s -c 300 -C aut…' terminated by signal SIGSEGV (Address boundary error)

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

Include any warning/errors/backtraces from the system logs

I'll resume the creation of my borg repository from the last checkpoint and I'll report back if I face more segfaults. Meanwhile, I also found a coredump. Here's the output from systemd-coredump.

Stack trace of thread 217360:
#0  0x000076c73c4a53f4 n/a (libc.so.6 + 0x963f4)
#1  0x000076c73c44c120 raise (libc.so.6 + 0x3d120)
#2  0x000076c73c44c1d0 n/a (libc.so.6 + 0x3d1d0)
#3  0x000076c73a50fcc0 n/a (checksums.cpython-312-x86_64-linux-gnu.so + 0x6cc0)
#4  0x000076c73c7a500d PyObject_Vectorcall (libpython3.12.so.1.0 + 0x1a500d)
#5  0x000076c73c789d71 _PyEval_EvalFrameDefault (libpython3.12.so.1.0 + 0x189d71)
#6  0x000076c73c84e395 PyEval_EvalCode (libpython3.12.so.1.0 + 0x24e395)
#7  0x000076c73c87263a n/a (libpython3.12.so.1.0 + 0x27263a)
#8  0x000076c73c86d4ef n/a (libpython3.12.so.1.0 + 0x26d4ef)
#9  0x000076c73c887b14 n/a (libpython3.12.so.1.0 + 0x287b14)
#10 0x000076c73c8873a1 _PyRun_SimpleFileObject (libpython3.12.so.1.0 + 0x2873a1)
#11 0x000076c73c886aff _PyRun_AnyFileObject (libpython3.12.so.1.0 + 0x286aff)
#12 0x000076c73c87f2c4 Py_RunMain (libpython3.12.so.1.0 + 0x27f2c4)
#13 0x000076c73c839eac Py_BytesMain (libpython3.12.so.1.0 + 0x239eac)
#14 0x000076c73c434e08 n/a (libc.so.6 + 0x25e08)
#15 0x000076c73c434ecc __libc_start_main (libc.so.6 + 0x25ecc)
#16 0x00005cb8329d4045 _start (python3.12 + 0x1045)
ELF object binary architecture: AMD x86-64

I can upload and link the coredump file if it helps. Let me know if I should.

[dionisio-bot[bot]]

Looks like this PR is not ready to merge, because of the following issues:

• This PR is missing the 'stat: QA assured' label
• This PR is missing the required milestone or project

Please fix the issues and try again

If you have any trouble, please check the PR guidelines

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: b309d567ebb2b1ca828bfb5fb6f2ca493d52b6c6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 34 packages

Name	Type
@rocket.chat/message-parser	Minor
@rocket.chat/meteor	Patch
@rocket.chat/core-services	Patch
@rocket.chat/core-typings	Patch
@rocket.chat/gazzodown	Major
@rocket.chat/livechat	Patch
@rocket.chat/rest-typings	Patch
rocketchat-services	Patch
@rocket.chat/account-service	Patch
@rocket.chat/authorization-service	Patch
@rocket.chat/ddp-streamer	Patch
@rocket.chat/omnichannel-transcript	Patch
@rocket.chat/presence-service	Patch
@rocket.chat/queue-worker	Patch
@rocket.chat/stream-hub-service	Patch
@rocket.chat/network-broker	Patch
@rocket.chat/omnichannel-services	Patch
@rocket.chat/presence	Patch
@rocket.chat/uikit-playground	Patch
@rocket.chat/api-client	Patch
@rocket.chat/apps	Patch
@rocket.chat/cron	Patch
@rocket.chat/ddp-client	Patch
@rocket.chat/freeswitch	Patch
@rocket.chat/fuselage-ui-kit	Patch
@rocket.chat/model-typings	Patch
@rocket.chat/ui-contexts	Patch
@rocket.chat/license	Patch
@rocket.chat/pdf-worker	Patch
@rocket.chat/models	Patch
@rocket.chat/ui-avatar	Patch
@rocket.chat/ui-video-conf	Patch
@rocket.chat/web-ui-registration	Patch
@rocket.chat/instance-status	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[Barrylimarti]

I don't know if this PR requires a changeset. Please let me know if this is needed as it is a change in the message parser package.

[rique223]

I don't know if this PR requires a changeset. Please let me know if this is needed as it is a change in the message parser package.

Hey Barrylimarti, first of all thank you so much for your PR. Yes, this PR is adding features to our existing systems so a changeset is necessary. Please refer to our documentation on development workflow, at the end of it there is a nice guideline on how to write changesets: https://developer.rocket.chat/docs/development-workflow

Also please have a look at our PR title guidelines: https://developer.rocket.chat/v1/docs/pull-requests-tags

[Barrylimarti]

Hi @tassoevan Thanks for approving the PR, although I saw there was one more linting error which was failing some cases. I committed the change after fixing it now. You can check it now.

[Barrylimarti]

Hey guys, I can see some of the checks failing in CI. Does this mean some test cases are not passing and Can you help me how to deal with these?

[Barrylimarti]

Hi @tassoevan Can you please review this again, as I checked everything on my system and the linting checks and test cases pass perfectly. It would be great motivation for me if this PR gets merged.