operations icon indicating copy to clipboard operation
operations copied to clipboard
verified publisher icon openstreetmap

Replace wal-g for postgresql wal archiving

Open Firefishy opened this issue 2 months ago • 4 comments

Currently we use wal-g for OpenStreetMap.org database archiving.

We need to decide and switch to a supported replacement.

Likely options:

  • pgbackreset
  • pg barman

Firefishy avatar Oct 05 '25 19:10 Firefishy

Which version of wal-g do we have? The project is still active so I'm surprised it doesn't work with PG 17.

pnorman avatar Oct 05 '25 20:10 pnorman

Reading through various talk slides it looks like s3 support is DIY for barman but built in for pgbackrest. pgbackrest seems to have better parallelism on large databases.

barman seems to handle multiple clusters better.

On the postgresql IRC channel pgbackrest is recommended more often than barman (135 times vs 6 times this year)

Given our usecase involve a very large database and WAL on S3, it sounds like pgbackrest is the better option for us.

pnorman avatar Oct 05 '25 21:10 pnorman

Which version of wal-g do we have? The project is still active so I'm surprised it doesn't work with PG 17.

There are no releases of wal-g that supports PG15+ https://github.com/wal-g/wal-g/pull/1909 , the issue was first reported back in 2022.

Firefishy avatar Oct 06 '25 09:10 Firefishy

I've created a S3 bucket and access key for dev to test alternatives: https://github.com/openstreetmap/operations/issues/1299

Firefishy avatar Dec 05 '25 00:12 Firefishy