server: support compressed MySQL protocol

[dveeden]

What problem does this PR solve?

Issue Number: close #22605

Problem Summary: Support deflate compression in the mysql protocol

This helps with:

• Performance: Especially data intensive work over low bandwidth connections
• Cost: This can reduce cost when bandwidth is payed for, e.g. in Cloud environments.

What is changed and how it works?

Check List

Tests

• [ ] Unit test
• [ ] Integration test
• [x] Manual test (add detailed scripts or steps below)
• [ ] No code

Side effects

• [ ] Performance regression: Consumes more CPU
• [ ] Performance regression: Consumes more Memory
• [ ] Breaking backward compatibility

Documentation

• [ ] Affects user behaviors
• [ ] Contains syntax changes
• [ ] Contains variable changes
• [ ] Contains experimental features
• [x] Changes MySQL compatibility

Release note

Support for the the compressed version of the MySQL protocol was added

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• hawkingrei
• lance6716

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment. After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review. Reviewer can cancel approval by submitting a request changes review.

[dveeden]

/cc @bb7133 @xhebox

[dveeden]

Tested with:

• mysql --compress
• mysqlsh --compress
• MySQL Connector/Python with mysql.connector.connect(compress=True, ...)

All the above with 8.x versions

[sre-bot]

Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/04b0a5f9bf70136a12e06daf2f35c8b61889b5fe

[dveeden]

Tested with:

• MySQL Connector/J 8.0.30 and useCompression=true
• MySQL Client 8.0.30 and mysql --compression-algorithms=zlib,zstd --zstd-compression-level=9 ...
• MySQL Shell 8.0.29 and mysqlsh --compression-algorithms=zstd --sql mysql://...
• MySQL Connector/Python 8.0.21 and mysql.connector.connect(compress=True, ...)

[dveeden]

/run-common-test

[lance6716]

/cc @lance6716

[dveeden]

With my latest update it now compresses multiple mysql packets into a single compressed packet once Flush() is called.

This works somewhat:

$ mysql -h 127.0.0.1 -u root -P 4000 --ssl-mode=DISABLED --compression-algorithms=zlib 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 405
Server version: 5.7.25-TiDB-v6.7.0-alpha-66-gc67761448-dirty TiDB Server (Apache License 2.0) Community Edition, MySQL 5.7 compatible

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> \s
ERROR 2013 (HY000): Lost connection to MySQL server during query
No connection. Trying to reconnect...
Connection id:    407
Current database: *** NONE ***

--------------
mysql  Ver 8.0.32 for Linux on x86_64 (MySQL Community Server - GPL)

Connection id:		407
Current database:	
Current user:		[email protected]
SSL:			Not in use
Current pager:		stdout
Using outfile:		''
Using delimiter:	;
Server version:		5.7.25-TiDB-v6.7.0-alpha-66-gc67761448-dirty TiDB Server (Apache License 2.0) Community Edition, MySQL 5.7 compatible
Protocol version:	10
Connection:		127.0.0.1 via TCP/IP
ERROR 2013 (HY000): Lost connection to MySQL server during query
No connection. Trying to reconnect...
Connection id:    409
Current database: *** NONE ***

Server characterset:	utf8mb4
Db     characterset:	utf8mb4
Client characterset:	utf8mb4
Conn.  characterset:	utf8mb4
TCP port:		4000
Protocol:		Compressed
Binary data as:		Hexadecimal
--------------

mysql> 

Next up is to figure out where we violate the protocol and fix that.

[dveeden]

This needs some tests, but otherwise it works.

Tested with:

• MySQL Client 8.0.33
• MySQL Shell 8.0.33
• MySQL Workbench 8.0.33
• MySQL Connector/Python 8.0.33 (with a fix for https://bugs.mysql.com/bug.php?id=110879)
• PHP 8.2.5 mysqlnd with PDO::MYSQL_ATTR_COMPRESS

This is with --compression-algorithms=zstd, --compression-algorithms=zlib and --compression-algorithms=zstd,zlib for MySQL Client and MySQL Shell. Other tools only seems to support zlib and not zstd.

The go-sql-driver/mysql doesn't support compression yet: https://github.com/go-sql-driver/mysql/issues/24

Wireshark (master branch/4.1) supports decompressing both zlib and zstd and can be used to check the client and server flags etc.

[dveeden]

/test unit-test

[dveeden]

/merge

[ti-chi-bot[bot]]

This pull request has been accepted and is ready to merge.

Commit hash: fb55f65a48022f0c606552bacbf59cba361555f4