Can "borg config [REPO] max_segment_size [VALUE]" be applied to existing repos?

[trendymail]

Hello!

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yep! Doc, FAQ and issues are a wealth of knowledge. :)

Is this a BUG / ISSUE report or a QUESTION?

QUESTION (maybe already asked but found nothing in the docs)

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

borg 1.2.6 (using "fat" binary "borg-linuxold64"

Operating system (distribution) and version.

OpenVZ/Virtuozzo 7

Hardware / network configuration, and filesystems used.

Dedicated servers, NVMe, ECC RAM, EXT4

How much data is handled by borg?

~2 To

Full borg commandline that lead to the problem (leave away excludes and passwords)

Not tested yet.

50 MiB per segment file

for REPO in $REPO_LIST; do borg config $REPO max_segment_size 52428800; done

Describe the problem you're observing.

I would like to understand and know:

a) if it is possible to apply this change to existing repos b) when "old" segment files (500 MiB) will be removed

Long story short:

• I use rsync to copy repos on low hardware systems
• I cannot use "borg create" on these secondary systems
• I fully understand that corruption, if any, will propagate

My goad is to limit rsync bandwidth usage sine all data is already compressed.

Is there any indication against using a lower "max_segment_size"? (in my case: 100 MiB)

For completeness, I use Borg to save OpenVZ disk images (results are awesome...).

Any hint will be greatly appreciated! ^^

[ThomasWaldmann]

Yes, you can apply this to existing repos, but it won't change the size of already existing segment files.

But: if such a segment file would get compacted (because of a above-threshold amount of deleted objects in it), the remaining objects in it would be written to new segment file(s) and for the new segment file(s), the new size limit would be obeyed.

[ThomasWaldmann]

Using 50 or 100MB segment size is perfectly reasonable, esp. for remote repos or repos that get rsynced to somewhere.

[trendymail]

Many, many thanks for your answers! :)

Unfortunately, I have just read tonight (in "Borg config" doc) that only local repos are allowed to use "borg config" command.

All my repos are "remote" (using SSH to access them), even if remote host is "localhost".

Authorized_keys entries (on remote hosts) always begin with "command="/usr/local/bin/borg serve",restrict ssh-rsa public key file"

And I do not plain to enable full SSH access to remote repos, even for non-root users.

---

To be more precise, I need to read (again) documentation to correctly understand "threat model" of Borg.

Meanwhile, feel free to "throw" me a RTFM link! ^^

[ThomasWaldmann]

You can also carefully edit repo/config, it is a text file.

[trendymail]

Understood.

But editing a remote config file would require having an SSH shell after each "borg init" to update max_segment_size parameter.

I would like to only depends on "borg serve" (if this is possible).

Goal is to be compatible with :

1. https://www.borgbase.com : the API does not provide a way to set "max_segment_size" (but I need to check this)
2. https://www.rsync.net/products/borg.html : borg client cannot set "max_segment_size" (If I am not wrong)

Maybe you know other providers that could deal with this (minor) "issue"?

---

Luckily, I do not use theses services so I can use SSH to manually modify each repo config file.

Question: would it be a "threat" to allow modifying "max_segment_size" option on insecure remote host?

As I said before, I must double-read the documentation to fully understand borg threat model.

Have a great day! :)

[ThomasWaldmann]

Yeah, it's a bit complicated.

The main problem there is permissions:

• there are some settings in the config that the repo hoster might not want to let the repo user edit, like e.g. quota settings.
• there are also config settings that the legitimate borg user would not want to let anyone client-side edit, like append-only setting (in case attacker gains access to borg client).
• other settings, like segment file size might be rather uncritical.

That's why I did not implement a RPC api back then for borg config. Maybe it would have been better to just use an editor in general and not have borg config at all.

[trendymail]

Again, many thanks for your valuable answers!

Would it be possible/acceptable/safe to add parameters to "borg init" : "--segments_per_dir" and "--max_segment_size"?

If these parameters are not "critical" and do not conflict with hosters (rsync, borgbase) policy, this would be great.

Have a great evening. :)

[ThomasWaldmann]

Theoretically yes. But for that working with remote repos, the repo server side borg would also need to support that.

[trendymail]

Understood.

Feel free to close this issue (question).

Have a great day/night. :)

[trendymail]

Long story short

it tuns out that:

• mirroring (with rsync) Borg repositories (~1.8 To),
• with default parameters (max_segment_size => 500 MB),
• on very low hardware system (Atom D525 2c/4t 1.80GHz 4 Go),
• and limited download bandwidth (35 Mbit/s),
• works well... :)

First rsync takes time (this is expected) but next syncs run quite fast (until borg prune and compact "reorder" chunk files).

Closing this question as resolved!

[trendymail]

Sorry, I should not have closed this "issue".

@ThomasWaldmann: Borg's documentation is incredible, may this question be helpful to you. :)