Barman stop WAL archiving after creating empty file

[ch9hn]

We are running into an issue with the archiving of WAL files with cloudnative-pg. In the very beginning Barman writes WAL files to the S3 Object Bucket and is then creating a plain file with the folder name inside the Bucket. After that no write happens again to the Bucket.

We are getting the following errors:

{"level":"info","ts":"2024-07-16T08:51:49Z","logger":"wal-archive","msg":"Failed archiving WAL: PostgreSQL will retry","logging_pod":"airflow-pg-1","walName":"pg_wal/00000006.history","startTime":"2024-07-16T08:51:48Z","endTime":"2024-07-16T08:51:49Z","elapsedWalTime":0.304129655,"error":"unexpected failure invoking barman-cloud-wal-archive: exit status 4"}
{"level":"info","ts":"2024-07-16T08:51:50Z","logger":"barman-cloud-wal-archive","msg":"2024-07-16 08:51:50,201 [80291] WARNING: Failed to parse headers (url=https://xxxxxx.s3.fr-par.scw.cloud:443/scw-cnpg-dev-stg-testing/airflow-pg/wals/0000000300000000/000000030000000000000010.partial.gz): [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 400 Bad request\\r\\nContent-length: 90\\r\\nCache-Control: no-cache\\r\\nConnection: close\\r\\nContent-Type: text/html\\r\\n\\r\\n'","pipe":"stderr","logging_pod":"airflow-pg-1"}
{"level":"info","ts":"2024-07-16T08:51:50Z","logger":"barman-cloud-wal-archive","msg":"Traceback (most recent call last):","pipe":"stderr","logging_pod":"airflow-pg-1"}
{"level":"info","ts":"2024-07-16T08:51:50Z","logger":"barman-cloud-wal-archive","msg":"  File \"/usr/local/lib/python3.9/dist-packages/urllib3/connectionpool.py\", line 487, in _make_request","pipe":"stderr","logging_pod":"airflow-pg-1"}
{"level":"info","ts":"2024-07-16T08:51:50Z","logger":"barman-cloud-wal-archive","msg":"    assert_header_parsing(httplib_response.msg)","pipe":"stderr","logging_pod":"airflow-pg-1"}
{"level":"info","ts":"2024-07-16T08:51:50Z","logger":"barman-cloud-wal-archive","msg":"  File \"/usr/local/lib/python3.9/dist-packages/urllib3/util/response.py\", line 91, in assert_header_parsing","pipe":"stderr","logging_pod":"airflow-pg-1"}
{"level":"info","ts":"2024-07-16T08:51:50Z","logger":"barman-cloud-wal-archive","msg":"    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)","pipe":"stderr","logging_pod":"airflow-pg-1"}
{"level":"info","ts":"2024-07-16T08:51:50Z","logger":"barman-cloud-wal-archive","msg":"urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 400 Bad request\\r\\nContent-length: 90\\r\\nCache-Control: no-cache\\r\\nConnection: close\\r\\nContent-Type: text/html\\r\\n\\r\\n'","pipe":"stderr","logging_pod":"airflow-pg-1"}
{"level":"info","ts":"2024-07-16T08:51:50Z","logger":"barman-cloud-wal-archive","msg":"2024-07-16 08:51:50,204 [80291] ERROR: Barman cloud WAL archiver exception: An error occurred () when calling the PutObject operation: ","pipe":"stderr","logging_pod":"airflow-pg-1"}

Before delete of the file:

After delete: As you see, a new WAL file is written and after that the write stops again.