wal-g icon indicating copy to clipboard operation
wal-g copied to clipboard

Allow point-in-time-restores of MySQL servers with encrypted binlogs

Open jstaf opened this issue 2 years ago • 1 comments

Database name

MySQL

Pull request description

Describe what this PR fixes

mysqlbinlog cannot use MySQL keyring plugins, which makes it impossible for it to read encrypted MySQL binary logs (using either encrypt_binlog=ON in Percona Server 5.7 or binlog_encryption=ON in MySQL 8.0). Though there's a python script out there that can decrypt binary logs created by MySQL 8.0, no tools are available to decrypt binlogs created by Percona Server 5.7 (or early versions of Percona Server 8.0). Since mysqlbinlog cannot decrypt encrypted binlogs on its own, there's currently no way to use WAL-G perform a point-in-time-restore if the MySQL server is using encrypted binlogs.

However, there is a workaround: MySQL servers using binlog encryption send the decrypted binlog to their replicas as part of MySQL replication. Likewise mysqlbinlog --raw --read-from-remote-server also fetches the decrypted binlog from an active server (the catch is that it saves the decrypted binlog to the working directory). This PR adds the ability to have WAL-G directly read unencrypted MySQL binlogs from a remote server the same way mysqlbinlog --raw --read-from-remote-server does without saving it to disk (the binary log still gets encrypted by WAL-G before sending it to cloud storage). The new option to read decrypted binary logs from the server is WALG_MYSQL_BINLOG_READ_FROM_REMOTE_SERVER. This lets us perform PITR restores for MySQL servers with encrypted binlogs as normal.

I also did a general documentation update for MySQL and documented how to perform backup and restores of encrypted tables as well as use the new WALG_STREAM_SPLITTER_PARTITIONS feature.

Please provide steps to test this PR

Try encrypting some tables with your favorite MySQL keyring plugin (keyring_file is the easiest to setup) and use the documentation in this PR to perform a backup and restore of those encrypted tables. You can try deleting the keyring file after taking a backup to prove that the instructions still work even if the original keyring has been lost.

I have personally tested this on MySQL 8.0 and Percona Server 5.7.

jstaf avatar Jul 22 '22 19:07 jstaf

Yes, that is a good idea on the tests (comparing binlog checksums was actually how I was initially testing this out). In this case we'd need to compare against the binlogs created by mysqlbinlog --raw --read-from-remote-server as the binlogs directly from mysqld will have different checksums since they're still encrypted. I will see if I can add some tests and address the rest of the comments early next week.

jstaf avatar Jul 27 '22 03:07 jstaf