Expire Command brings ERROR: [042] ERROR: [104]

[wolfgangbrandl]

Please provide the following information when submitting an issue (feature requests or general comments can skip this):

1. pgBackRest version: pgBackRest 2.38

2. PostgreSQL version: 13.7

3. Operating system/version - if you have more than one server (for example, a database server, a repository host server, one or more standbys), please specify each: Red Hat Enterprise Linux Server release 7.9 (Maipo)

4. Did you install pgBackRest from source or from a package? yes

5. Please attach the following as applicable:

   • pgbackrest.conf file(s)
   • postgresql.conf settings applicable to pgBackRest (archive_command, archive_mode, listen_addresses, max_wal_senders, wal_level, port)
   • errors in the postgresql log file before or during the time you experienced the issue
   • log file in /var/log/pgbackrest for the commands run (e.g. /var/log/pgbackrest/mystanza_backup.log)

6. Describe the issue: We execute the following command on the leader of a patroni cluster: pgbackrest --log-level-console=info --config-path=/products/pgbackrest/etc --stanza=pgclust --process-max=1 --type=full backup and got the following error in the expire command WARN: environment contains invalid option 'home' 2022-07-30 22:15:09.473 P00 INFO: backup command begin 2.38: --config-path=/products/pgbackrest/etc --exec-id=3543985-bd5c886c --log-level-console=info --log-level-file=info --log-path=/products/pgbackrest/log --pg1-path=/data/db/db0/PGHST --pg1-port=5432 --pg1-socket-path=/logs/db/db0/PGHST/logs --process-max=1 --repo1-path=/PGHST --repo1-retention-full=2 --repo1-s3-bucket=bu-pg-hst --repo1-s3-endpoint=s3.brz.gv.at --repo1-s3-key= --repo1-s3-key-secret= --repo1-s3-region=Standard --repo1-s3-uri-style=path --no-repo1-storage-verify-tls --repo1-type=s3 --stanza=pghst --start-fast --type=full 2022-07-30 22:15:11.311 P00 INFO: execute non-exclusive pg_start_backup(): backup begins after the requested immediate checkpoint completes 2022-07-30 22:15:11.812 P00 INFO: backup start archive = 0000000100000026000000E0, lsn = 26/E0000028 2022-07-30 22:15:11.812 P00 INFO: check archive for prior segment 0000000100000026000000DF 2022-07-30 22:17:50.747 P00 INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive 2022-07-30 22:17:50.948 P00 INFO: backup stop archive = 0000000100000026000000E0, lsn = 26/E0055D78 2022-07-30 22:17:51.279 P00 INFO: check archive for segment(s) 0000000100000026000000E0:0000000100000026000000E0 2022-07-30 22:17:51.596 P00 INFO: new backup label = 20220730-221511F 2022-07-30 22:17:52.175 P00 INFO: full backup size = 33.7MB, file total = 1514 2022-07-30 22:17:52.176 P00 INFO: backup command end: completed successfully (162704ms) 2022-07-30 22:17:52.176 P00 INFO: expire command begin 2.38: --config-path=/products/pgbackrest/etc --exec-id=3543985-bd5c886c --log-level-console=info --log-level-file=info --log-path=/products/pgbackrest/log --repo1-path=/PGHST --repo1-retention-full=2 --repo1-s3-bucket=bu-pg-hst --repo1-s3-endpoint=s3.brz.gv.at --repo1-s3-key= --repo1-s3-key-secret= --repo1-s3-region=Standard --repo1-s3-uri-style=path --no-repo1-storage-verify-tls --repo1-type=s3 --stanza=pghst 2022-07-30 22:17:53.136 P00 INFO: repo1: expire full backup set 20220716-221506F, 20220716-221506F_20220717-060011I, 20220716-221506F_20220718-060014I, 20220716-221506F_20220719-060010I, 20220716-221506F_20220720-060009I, 20220716-221506F_20220721-060008I, 20220716-221506F_20220722-060009I 2022-07-30 22:17:53.340 P00 INFO: repo1: remove expired backup 20220716-221506F_20220722-060009I 2022-07-30 22:17:57.372 P00 INFO: repo1: remove expired backup 20220716-221506F_20220721-060008I 2022-07-30 22:18:02.669 P00 INFO: repo1: remove expired backup 20220716-221506F_20220720-060009I 2022-07-30 22:18:08.315 P00 INFO: repo1: remove expired backup 20220716-221506F_20220719-060010I 2022-07-30 22:18:14.582 P00 INFO: repo1: remove expired backup 20220716-221506F_20220718-060014I 2022-07-30 22:18:17.775 P00 INFO: repo1: remove expired backup 20220716-221506F_20220717-060011I 2022-07-30 22:18:20.296 P00 INFO: repo1: remove expired backup 20220716-221506F ERROR: [042]: repo1: unexpected eof while reading line ERROR: [104]: expire command encountered 1 error(s), check the log file for details 2022-07-30 22:19:20.477 P00 INFO: expire command end: aborted with exception [104]

pgbackrest-issue.zip

[dwsteele]

There's not a lot to go on here, but the error appears to have been transient since the next expire succeeds.

I suspect that this situation would be helped by this planned improvement: https://github.com/pgbackrest/pgbackrest/projects/2#card-41839856.