MySQL Binlog Transaction Compression

[altmannmarcelo]

This adds support for MySQL Binlog Transaction Compression WL https://dev.mysql.com/worklog/task/?id=3549

The main change is how the adaptor processes ReplicationAction events. It now receives a vector of events and processes them in order. MySQL and PG connectors have been adjusted to return a vector of a single event. When MySQL receives a transaction payload consisting of multiple events with the same replication position, it returns the actionable events into the vector.

This requires mysql-common with support for the TRANSACTION_PAYLOAD event. Probably 0.30.7 (not yet released). Alternatively, the list of commits below can be cherry-picked on top of the current 0.29.2 :

• https://github.com/blackbeam/rust_mysql_common/commit/22f0a559aea133c6f63ef5709e3d1a494243e36c
• https://github.com/blackbeam/rust_mysql_common/commit/b6160077fd90c844f2ced05b4955bf94cb01c687
• https://github.com/blackbeam/rust_mysql_common/commit/7312eeccd676b9b5273232a8e55d6398c85ea46a
• https://github.com/blackbeam/rust_mysql_common/commit/1470f8e09db295647d6aeb304d88e295f25c2ecf
• https://github.com/blackbeam/rust_mysql_common/commit/5713e6ddc5e5db637e91e43071ab00cd624bf413

For testing purposes, I have a branch with those commit on top of 0.29.2 at https://github.com/altmannmarcelo/rust_mysql_common/tree/rs-0.29.2-transaction-compression.

$ git diff Cargo.toml
diff --git a/Cargo.toml b/Cargo.toml
index 88530053b..263111d2b 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -4,6 +4,7 @@ eui48 = { git = "https://github.com/readysettech/eui48.git", branch = "master" }
 opentelemetry = { git = "https://github.com/open-telemetry/opentelemetry-rust" }
 opentelemetry-otlp = { git = "https://github.com/open-telemetry/opentelemetry-rust" }
 opentelemetry-semantic-conventions = { git = "https://github.com/open-telemetry/opentelemetry-rust" }
+mysql_common = { git = "https://github.com/altmannmarcelo/rust_mysql_common.git", branch = "rs-0.29.2-transaction-compression" }
 
 [workspace]
 members = [

Testing

1. On the upstream database, enable general log and check if the query is begin served directly from RS :

mysql> SET GLOBAL general_log = ON;
mysql> exit;
tail -f datadir1/marce-bld.log

2. On a separate session, connect to RS:

mysql -P 3307 --prompt='RS> '
RS> use test;
RS> CREATE TABLE user (ID INT PRIMARY KEY AUTO_INCREMENT, name varchar(100));
RS> INSERT INTO user VALUES (NULL, 'Marcelo');
RS> INSERT INTO user VALUES (NULL, 'Other');

3. Create the cache:

RS> CREATE CACHE FROM SELECT COUNT(*) FROM user WHERE name = 'Marcelo';

5. Run the cached query on RS and check general log. We should not see the query on the general log:

RS> SELECT COUNT(*) FROM user WHERE name = 'Marcelo';
+----------+
| count(*) |
+----------+
|        1 |
+----------+
1 row in set (0,00 sec)

RS> SELECT COUNT(*) FROM user WHERE name = 'Other';
+----------+
| count(*) |
+----------+
|        1 |
+----------+
1 row in set (0,01 sec)

6. Enable compression and create a multi-statement insert affecting the cached table.

RS> SET binlog_transaction_compression=ON;
RS> INSERT INTO user VALUES (NULL, 'Marcelo'), (NULL, 'Marcelo'), (NULL, 'Other');
RS> COMMIT;

7. Run the cached query on RS. We should see the updated values and no new query on general log:

RS> SELECT COUNT(*) FROM user WHERE name = 'Marcelo';
+----------+
| count(*) |
+----------+
|        3 |
+----------+
1 row in set (0,00 sec)

RS> SELECT COUNT(*) FROM user WHERE name = 'Other';
+----------+
| count(*) |
+----------+
|        2 |
+----------+
1 row in set (0,00 sec)