borg icon indicating copy to clipboard operation
borg copied to clipboard

Borg "fat" binaries extract "many" files before running: should TMPDIR be a tmpfs?

Open trendymail opened this issue 1 year ago • 4 comments

Hello!

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yup! Docs, FAQ and Issues.

Every time I read an issue, I learn something... ^^

Is this a BUG / ISSUE report or a QUESTION?

Just a question

Your borg version (borg -V).

borg 1.2.7 ("fat") binaries => borg-linuxold64 and borg-linux64.

Describe the problem you're observing.

Not really a problem, I think that "fat" binaries need to extract some files to be "compatible".

Long story short

Each time a "fat" binary is launched, it extracts some data in order to run (so far so good).

But, can these needed files be stored on an tmpfs?

I do actually set TMPDIR to a tmpfs dir.

Results are (very) good: launch time is divided by 2.

But, since TMPDIR is needed for other files, is 1 GB is enough?

----

Please simply send me a "RTFM" if I missed something. ^^

Have a great night! :)

trendymail avatar Dec 31 '23 04:12 trendymail

I suppose no one has counted how much space needed for this, but

https://borgbackup.readthedocs.io/en/stable/installation.html#standalone-binary

Note that the binary uses /tmp to unpack Borg with all dependencies. It will fail if /tmp has not enough free space or is mounted with the noexec option. You can change the temporary directory by setting the TEMP environment variable before running Borg.

infectormp avatar Dec 31 '23 05:12 infectormp

There are 2 borg "fat binary" builds:

  • the one without extension is just a single, compressed file
  • the *.tgz one is a directory full of files, but otherwise the same

If you extract the tgz, you will see the amount and total size of the extracted files for both cases.

If you care a lot for speed, use the tgz and just extract it once.

ThomasWaldmann avatar Jan 01 '24 14:01 ThomasWaldmann

@infectormp: many thanks for the RTFM link (I overlooked part "[...] unpack Borg with all dependencies."). @ThomasWaldmann: I was not aware that tgz files were "unpacked" versions of standalone binaries. ;)

Using tgz versions did not yield much performance differences but it allows to skip extraction each time Borg is called.

But hey, "Every Small Step Counts"... ^^

Thank you for your time and your answers, I really appreciate!

trendymail avatar Jan 19 '24 22:01 trendymail

TODO: add a note to the borg binaries 00_README.txt that the single file version will always extract to temp and that the tgz single directory version is already extracted and does not need to do that.

ThomasWaldmann avatar Jul 14 '24 13:07 ThomasWaldmann