False alarms from pgBackRest events

[kmatt]

Similar to [1], TnM is generating emails on INFO events logged in the PostgreSQL logs by pgBackRest. These events do not have any of the INCLUDE keywords, so I'm wondering if the format of the log entries is triggering emails, because the example below reporting LINE_TRUNCATED.

Attempts with EXCLUDE have not prevented the alerts. Changing the log_line_prefix in postgresql.conf has also been unsuccessful.

For example, note the event does not have a time zone (GMT) with the timestamp as standard Postgres log entries do (2020-07-16 15:00:04 GMT ...), this is an email from tail_n_mail, where there are no error conditions in the log:

[2] (from lines 5 - 14636)
2020-07-16 23:05:33.171 P00   INFO: pushed WAL file '000000010000430A000000E7' to the archive asynchronously
2020-07-16 23:05:33.171 P00   INFO: archive-push command end: completed successfully (102ms)
2020-07-16 23:05:33.924 P00   INFO: archive-push command begin 2.27: [pg_wal/000000010000430A000000E8] --archive-async --archive-timeout=90 --compress-type=lz4 --log-level-console=info --log-level-f
[LINE TRUNCATED, original was 651,208 characters and 3,007 lines long]

tnm.config:

LOG_LINE_PREFIX: %t P%p
MAILSUBJECT: TnM - HOST Postgres errors UNIQUE : NUMBER

INCLUDE: ERROR:
INCLUDE: FATAL:
INCLUDE: PANIC:
EXCLUDE: canceling autovacuum
EXCLUDE: INFO: archive-push
EXCLUDE: INFO: pushed

FILE1: /localdata/pgdata11/log/LATEST
LASTFILE1: /localdata/pgdata11/log/postgresql-20200716_145958.log

[1] https://bucardo.org/pipermail/tnm/2018-August/000046.html

[kmatt]

–skip_non_parsed stops the reporting of extra log entries, but still uncertain of why the pgBackRest events do not parse.

[kmatt]

Experimenting with making the Postgres log line prefix similar (%m) and the tail_n_mail timezone regex optional: https://github.com/bucardo/tail_n_mail/compare/master...kmatt:2.8.1-pgbackrest

Ideally changing the server log format would not be necessary.

[turnstep]

The pgbackrest lines do not parse because they do not respect log_line_prefix and just write their own format, which is a little rude. For now, --skip_non-parsed is indeed the best solution, but I will look into something better.