`--target-immediate` option copies all WAL files

[rkrzr]

trafficstars

The Problem

When restoring a backup with barman recover --target-immediate ... it does not only restore the specified base backup, but it also copies all WAL files that belong to that base backup. (See https://github.com/2ndquadrant-it/barman/commit/12ce71b35f8cbbf69331f2eef6cbc49754af15fc#diff-641b3483164effa2fc31a8ee8bed3da0R167)

This is unexpected, since none of those WAL files are actually necessary, since recovery should end as soon as a consistent state is reached (at the end of the base backup). (See this commit message: https://github.com/2ndquadrant-it/barman/commit/12ce71b35f8cbbf69331f2eef6cbc49754af15fc#diff-641b3483164effa2fc31a8ee8bed3da0L145)

This behavior is problematic in our case, since we have much more data in WAL files, than in the base backup, so it can take hours to copy all those files (which are then not actually needed).

Expected behavior

The barman recover --target-immediate ... should not copy the WAL files and finish after restoring the base backup.

[ccppprogrammer]

Any news on this? After recovering the base backup with option barman recover --target-immediate

2020-11-15 12:28:11,488 [28903] barman.copy_controller INFO: Copy finished (safe before None)

barman starts copying any WALs we don't need.

2020-11-15 12:28:12,250 [28903] barman.recovery_executor INFO: Copying required WAL segments.
2020-11-15 12:28:12,437 [28903] barman.recovery_executor INFO: Starting copy of 227 WAL files 227/9891 from WalFileInfo(compression='bzip2', name='000000010000025C0000001D', size=7333905, time=1602719902.9227433) to WalFileInfo(compression='bzip2', name='000000010000025C000000FF', size=7979981, time=1602748718.8567579)

# barman --version
2.11

# lsb_release -a
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic

[gbartolini]

That is a known limitation of the "push" method of WAL files. Barman always ships all the available WAL files from the start to the end of the WAL.

You need to look into enabling the "pull" method of WAL restore, which in Barman's terms means taking advantage of the "get-wal" option. Please look into that in the documentation and in 2ndQuadrant's blog (even though they are old they should explain):

https://www.2ndquadrant.com/en/blog/getting-wal-files-from-barman-with-get-wal/

https://www.2ndquadrant.com/en/blog/speed-up-getting-wal-files-from-barman/

Cheers, Gabriele

On Sun, 15 Nov 2020 at 12:37, sergei lavrov [email protected] wrote:

Any news on this? After recovering the base backup with option barman recover --target-immediate

2020-11-15 12:28:11,488 [28903] barman.copy_controller INFO: Copy finished (safe before None)

barman starts copying any WALs we don't need.

2020-11-15 12:28:12,250 [28903] barman.recovery_executor INFO: Copying required WAL segments. 2020-11-15 12:28:12,437 [28903] barman.recovery_executor INFO: Starting copy of 227 WAL files 227/9891 from WalFileInfo(compression='bzip2', name='000000010000025C0000001D', size=7333905, time=1602719902.9227433) to WalFileInfo(compression='bzip2', name='000000010000025C000000FF', size=7979981, time=1602748718.8567579)

barman --version

2.11

lsb_release -a

Distributor ID: Ubuntu Description: Ubuntu 18.04.4 LTS Release: 18.04 Codename: bionic

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/2ndquadrant-it/barman/issues/205#issuecomment-727555875, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABMPB7CK2YWFWGVTNOMWD3SP64PNANCNFSM4HCX56SQ .

-- Gabriele Bartolini - Head of Global Support PostgreSQL Training, Services and Support [email protected] | www.2ndQuadrant.com http://www.2ndquadrant.com/

[ccppprogrammer]

Hello Gabriele! Thanks for the quick reply! I know about the pull method and I use it in the production in many installations, but I'm talking about restoring barman's backup, for example on dev server where I don't need any WALs and where I don't want to specify --target-time option, I just need the specific base backup without corresponding WALs. Am I right that the barman doesn't have this option right now?

[gbartolini]

Hi,

On Sun, 15 Nov 2020 at 13:07, sergei lavrov [email protected] wrote:

Am I right that the barman doesn't have this option right now?

That's correct. It is a missing feature.

-- Gabriele Bartolini - Head of Global Support PostgreSQL Training, Services and Support [email protected] | www.2ndQuadrant.com http://www.2ndquadrant.com/