[wip] Proof-of-concept fix for missing .partial WALs on recovery

[mikewallace1979]

Updated with a different approach where .partial files are handled after copying the WALs in the archive.

There are a few upsides to doing this:

• We no longer need special cases in the code branch for compressed WALs.
• We can use Rsync.from_file_list to copy the .partial files when WALs are uncompressed, since all .partial files now get copied and renamed in a temporary staging directory.

Including the .partial WAL files in the get_required_xlog_files response seems a bit odd since one of the first things we do in _xlog_copy is put them in a separate list so they can be handled after the WALs in the archive are copied, however it feels like the right thing to do given that the .partial file is a required xlog file (unless the requested recovery timeline or time dictate otherwise).

[edb-sonar-app[bot]]

SonarQube Quality Gate:

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

35.7% Coverage
0.0% Duplication

[mikewallace1979]

Having talked this through with @amenonsen the assumption about the .partial file being uncompressed is reasonable.

Compression in the .partial file can happen in two places:

1. PostgreSQL wal_compression - if this is set then pages within the WAL will be compressed, however the file itself is not a compressed file in any sense that external tools need to care.
2. pg_receivewal compression - this compresses the .partial file but applies a .gz.partial suffix:

	/* File looks like a partial gzip-compressed WAL file */
	if (fname_len == XLOG_FNAME_LEN + strlen(".gz.partial") &&
		strcmp(filename + XLOG_FNAME_LEN, ".gz.partial") == 0)
	{
		*ispartial = true;
		*wal_compression_method = COMPRESSION_GZIP;
		return true;
	}

Given:

1. Barman runs pg_receivewal itself and we do not support it being run out-of-band writing files into the streaming directory.
2. Barman does not use pg_receivewal's compression feature.
3. Barman does not consider a .gz.partial file to be a WAL.

Any .partial file we encounter in Barman at this point is therefore going to be uncompressed (at the file level).

[edb-sonar-app[bot]]

SonarQube Quality Gate

0 Bugs
0 Vulnerabilities
0 Security Hotspots
2 Code Smells

46.5% Coverage
0.0% Duplication

[martinmarques]

@mikewallace1979 do you think this is something we would like to keep pursuing?