borg import-tar / create / recreate: add --strip-components

[kingIZZZY]

Trying to use borg import-tar

Docs say: a --tar-filter program may be explicitly specified. It should read compressed data from stdin and output an uncompressed tar stream on stdout.

Can someone please show me how to do tar -xzv --strip-components=1 as the filter program? I tried a few different ways but always just seems to hang..

borg import-tar --tar-filter="tar -xzvf - --strip-components=1" /path/to/repo::archive-name /path/to/something.tar.gz borg import-tar --tar-filter="tar -xzv --strip-components=1 -f -" /path/to/repo::archive-name /path/to/something.tar.gz etc..

Shouldn't -f - make tar read from stdin and output to stdout?

[ThomasWaldmann]

Can someone please show me how to do tar -xzv --strip-components=1 as the filter program?

No, because borg import-tar expects a stream in tar format, so you can't use an external tar command to already unpack the tar.

The --tar-filter option is usually used for decompressing stuff like filename.tar.<somecompression> and is not even needed for most cases because they are autodetected (like e.g. .tar.gz).

[kingIZZZY]

So if i need to do --strip-components=1 (i.e. skip the tar's root folder) then there's no other choice but to fully extract the whole tar as a separate step, then borg create?...

[ThomasWaldmann]

Stripping path components needs to be done inside borg, check if that is already implemented for import-tar.

[kingIZZZY]

Apparently not? I get an error that the option "--strip-components" is not recognized

I think it's implemented for export-tar but not import-tar... in my particular case it would be HUGELY useful if import-tar also had this option 🙏

[ThomasWaldmann]

Yeah, correct, --strip-components is not implemented for borg import-tar (and also not for borg create and not for borg recreate, #1650).

[kingIZZZY]

Could you please explain what do you mean it does not exist for borg create ? How might one use tar with borg create? And how would --strip-components be relevant/useful for borg create?

If there exists some other way how to extract a tar and use it with (pipe it to?) borg create - I would love to know how. That is - if it can archive the actual files & folders not just a single huge monolith "stdin" file with just raw piped data...

[ThomasWaldmann]

import-tar and create are doing similar things (creating a borg archive from either tar member items or from the fs).

[kingIZZZY]

I tried using borg create to make an archive from a tar file, but it seems to put all the data into a single "stdin" file Not properly expanded/extracted distinct files & folders This seems to be officially how it works from docs "... By default, the content read from stdin is stored in a file called ‘stdin’. Use --stdin-name to change the name. "

Unless i'm missing something?

[ThomasWaldmann]

borg create usually makes backups from your filesystem. if you pump data in via the stdin mode, it is not expected to do any processing on that (and it does not know what the stuff is).

for importing tar files, there's import-tar, but it misses the feature you want.

[kingIZZZY]

👌 ok thanks for clarifying about all that

Much looking forward to the day import-tar + --strip-components is working

[qupfer]

Not sure if I got the problem right, but strip-components is an tar "extraction" option. If your "input" contains the path, its part of the TAR. Nothing can done there.

BUT you should be able to "workaround" Lets say, we have this path: /my/long/path/on/the/tar/journey and only want journey+content. (see example below)

### to test ### 
cd /tmp
mkdir -p  /tmp/my/long/path/on/the/tar/journey
touch my/long/path/on/the/tar/journey/1
touch my/long/path/on/the/tar/journey/2
touch my/long/path/on/the/tar/journey/3

# tar directly
tar -cf test.tar /tmp/my/long/path/on/the/tar/journey/
tar -tf test.tar  
tmp/my/long/path/on/the/tar/journey/ # <-- BAD
tmp/my/long/path/on/the/tar/journey/1 # <-- BAD
tmp/my/long/path/on/the/tar/journey/2 # <-- BAD
tmp/my/long/path/on/the/tar/journey/3 # <-- BAD

# tar with -C
tar -cf test.tar -C /tmp/my/long/path/on/the/tar journey
tar -tf test.tar
journey/ # <-- GOOD
journey/1 # <-- GOOD
journey/2 # <-- GOOD
journey/3 # <-- GOOD

# tar with --transform (sed substitition syntax)
tar -cf test.tar --transform='s#tmp/my/long/path/on/the/tar/journey#journey#' /tmp/my/long/path/on/the/tar/journey/ 
### attention to us tmp and not /tmp as tar strip always the root-slash '/' --> not part of the filename during path-transformation ###
tar -tf test.tar
journey/ # <-- GOOD
journey/1 # <-- GOOD
journey/2 # <-- GOOD
journey/3 # <-- GOOD

And now we can use this for borg. Instead of "-f" wie use "-O" (for stdout output) tar -c -O -C /tmp/my/long/path/on/the/tar journey | borg import-tar [...] - or tar -c -O -C /tmp/my/long/path/on/the/tar journey | borg create [...] - (thats contain the tar unextracted as a single stdin File)

[ThomasWaldmann]

@qupfer we no not use the tar cli command for borg import-tar, but the tarfile module from python standard library.

[qupfer]

@qupfer we no not use the tar cli command for borg import-tar, but the tarfile module from python standard library.

Yes, what’s why I named it workaround. Using cli tar (uncompressed) to modify input. But I oversee that the existing input was already Tar. So I think he could (de-tar, re-tar, Borg) tar -xz .. | tar -c -C -O .. | borg import-tar Or import as it is and use borg export-tar | tar -x —strip-components ..