wal-g
wal-g copied to clipboard
Allow point-in-time-restores of MySQL servers with encrypted binlogs
Database name
MySQL
Pull request description
Describe what this PR fixes
mysqlbinlog
cannot use MySQL keyring plugins, which makes it impossible for it to read encrypted MySQL binary logs (using either encrypt_binlog=ON
in Percona Server 5.7 or binlog_encryption=ON
in MySQL 8.0). Though there's a python script out there that can decrypt binary logs created by MySQL 8.0, no tools are available to decrypt binlogs created by Percona Server 5.7 (or early versions of Percona Server 8.0). Since mysqlbinlog cannot decrypt encrypted binlogs on its own, there's currently no way to use WAL-G perform a point-in-time-restore if the MySQL server is using encrypted binlogs.
However, there is a workaround: MySQL servers using binlog encryption send the decrypted binlog to their replicas as part of MySQL replication. Likewise mysqlbinlog --raw --read-from-remote-server
also fetches the decrypted binlog from an active server (the catch is that it saves the decrypted binlog to the working directory). This PR adds the ability to have WAL-G directly read unencrypted MySQL binlogs from a remote server the same way mysqlbinlog --raw --read-from-remote-server
does without saving it to disk (the binary log still gets encrypted by WAL-G before sending it to cloud storage). The new option to read decrypted binary logs from the server is WALG_MYSQL_BINLOG_READ_FROM_REMOTE_SERVER
. This lets us perform PITR restores for MySQL servers with encrypted binlogs as normal.
I also did a general documentation update for MySQL and documented how to perform backup and restores of encrypted tables as well as use the new WALG_STREAM_SPLITTER_PARTITIONS
feature.
Please provide steps to test this PR
Try encrypting some tables with your favorite MySQL keyring plugin (keyring_file
is the easiest to setup) and use the documentation in this PR to perform a backup and restore of those encrypted tables. You can try deleting the keyring file after taking a backup to prove that the instructions still work even if the original keyring has been lost.
I have personally tested this on MySQL 8.0 and Percona Server 5.7.
Yes, that is a good idea on the tests (comparing binlog checksums was actually how I was initially testing this out). In this case we'd need to compare against the binlogs created by mysqlbinlog --raw --read-from-remote-server
as the binlogs directly from mysqld
will have different checksums since they're still encrypted. I will see if I can add some tests and address the rest of the comments early next week.