readyset
readyset copied to clipboard
Published
20 hours ago •
readysettech
readysettech
WIP: Experimenting with Vitess support for ReadySet
This is an experiment. Trying to add Vitess support to ReadySet.
Tentative plan:
- [x] Generate a GRPC client for Vitess APIs (done in https://crates.io/crates/vitess-grpc)
- [x] Implement data parsing for VStream API events (conversion to Noria data types)
- [x] Implement Vitess schema caching from Field events
- [x] Implement MySQL and Vitess replication position tracking
- [x] Snapshots support
- [x] Get schema for all tables and from Vitess (using the same approach
MySQLReplicatoruses by connecting to MySQL port on VTgate) - [x] Data snapshot is via VStream Copy
- [x] Implement the basic version by setting the initial VGTID to an empty value, triggering VSteram Copy and receiving all the data.
- [x] IDEA: Parallel dumping of tables by running a VStream Copy process with a single table filter until
COPY_COMPLETED. That would give us a VGTID for each table. Then we can start with min(table VGTID) and catch up to max(table VGTID). Finally, use the max(GTID) as the position to start replicating from.- Single table VStream Copy example:
grpcurl -plaintext -d '{"vgtid":{"shard_gtids":[ { "keyspace":"commerce" }]}, "filter":{"rules":[{"match":"shirts"}]}}' localhost:15301 vtgateservice.Vitess.VStream
- Single table VStream Copy example:
- [ ] Parallel table snapshotting
- [x] Get schema for all tables and from Vitess (using the same approach
- Implement a connector for VStream API
- [x] Base scaffolding
- [x] Position tracking (VGTID VEvent)
- [x] Position resume
- [x] Schema tracking (FIELD VEvent)
- [x] DDL operations (DDL VEvent)
- [x] Ensure buffered rows are flushed before a DDL is processed (probably the same way Postgres does it via the "peek" event idea).
- [x] DML operations (ROW VEvent)
- [x] Insert
- [x] Update
- [x] Delete
- [x] Ensure that row event buffer never contains events from more than one table (see how Postgres connector uses the "peek" event idea to postpone processing of an event until the next call)
- [x] Connection handler for Vitess
- Reuse the existing MySqlHandler going through the vtgate mysql port.
- [x] LazyAdapter wrapper (https://github.com/readysettech/readyset/issues/496)
- [ ] Address TODOs in code
- [ ]
database-utils - [ ]
readyset-logictest - [ ]
readyset-clustertest - [ ]
benchmarks - [ ]
system-benchmarks
- [ ]
- [ ] Investigate switching gRPC for MySQL backend calls (to avoid having to speak MySQL protocol to Vitess and simplify configuration).
- ...
An example single-table VStream Copy:
{
"events": [
{
"type": "BEGIN",
"keyspace": "commerce",
"shard": "0"
},
{
"type": "FIELD",
"fieldEvent": {
"tableName": "commerce.customer",
"fields": [
{
"name": "customer_id",
"type": "INT64",
"table": "customer",
"orgTable": "customer",
"database": "vt_commerce_0",
"orgName": "customer_id",
"columnLength": 20,
"charset": 63,
"flags": 49667
},
{
"name": "email",
"type": "VARCHAR",
"table": "customer",
"orgTable": "customer",
"database": "vt_commerce_0",
"orgName": "email",
"columnLength": 512,
"charset": 255
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
}
]
}
{
"events": [
{
"type": "VGTID",
"vgtid": {
"shardGtids": [
{
"keyspace": "commerce",
"shard": "0",
"gtid": "MySQL56/65f53702-66ed-11ee-a051-3ec26fc62fce:1-53"
}
]
},
"keyspace": "commerce",
"shard": "0"
}
]
}
{
"events": [
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"1",
"19"
],
"values": "MTgxNjMxY2ExQDY0NGJiNC5jb20="
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"1",
"19"
],
"values": "Mjg0NDNiZjMzQDEzZGQ2Mi5jb20="
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"1",
"19"
],
"values": "Mzc4Nzc4MDliQGRkZGRlZC5jb20="
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"1",
"19"
],
"values": "NGE3N2MxM2U5QDg1N2IyNS5jb20="
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"1",
"19"
],
"values": "NWUxODMyYWY0QGIwNjI0NC5jb20="
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"1",
"19"
],
"values": "NjliY2EzMWMwQDkxYmM5Ni5jb20="
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"1",
"19"
],
"values": "N2ZmODc2NmJhQDY1YWEzMi5jb20="
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"1",
"19"
],
"values": "ODk3ZGI0OGM5QGY1ZjlkZC5jb20="
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"1",
"19"
],
"values": "OTkwNGM4ZGU5QDNiZDc4Mi5jb20="
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"2",
"19"
],
"values": "MTBiNjBlNDhhYkA2NTg5NGEuY29t"
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"2",
"19"
],
"values": "MTFlM2ZlMmEzYkAwYjZiMzEuY29t"
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"2",
"19"
],
"values": "MTI4NzVlODk1NUA2OWQyNmQuY29t"
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"2",
"19"
],
"values": "MTNmMTM4MmExYUAzMjNlZjYuY29t"
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"2",
"19"
],
"values": "MTRlZjcxNGI2N0BmYjcwNDEuY29t"
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"2",
"19"
],
"values": "MTUxMzc0ODE0Y0AyZjcyMzYuY29t"
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"2",
"19"
],
"values": "MTYyM2I2ZThiM0AwYWI1OTYuY29t"
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"2",
"19"
],
"values": "MTc2OGU0MTQ1MEAzM2U4NGQuY29t"
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"2",
"19"
],
"values": "MTg4NDU0YzlhM0BkYmU0ZDIuY29t"
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "ROW",
"rowEvent": {
"tableName": "commerce.customer",
"rowChanges": [
{
"after": {
"lengths": [
"2",
"19"
],
"values": "MTljMDkxNzQwN0BkMmIwNTYuY29t"
}
}
],
"keyspace": "commerce",
"shard": "0"
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "VGTID",
"vgtid": {
"shardGtids": [
{
"keyspace": "commerce",
"shard": "0",
"gtid": "MySQL56/65f53702-66ed-11ee-a051-3ec26fc62fce:1-53",
"tablePKs": [
{
"tableName": "customer",
"lastpk": {
"fields": [
{
"name": "customer_id",
"type": "INT64"
}
],
"rows": [
{
"lengths": [
"2"
],
"values": "MTk="
}
]
}
}
]
}
]
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "COMMIT",
"keyspace": "commerce",
"shard": "0"
}
]
}
{
"events": [
{
"type": "BEGIN",
"keyspace": "commerce",
"shard": "0"
},
{
"type": "VGTID",
"vgtid": {
"shardGtids": [
{
"keyspace": "commerce",
"shard": "0",
"gtid": "MySQL56/65f53702-66ed-11ee-a051-3ec26fc62fce:1-53"
}
]
},
"keyspace": "commerce",
"shard": "0"
},
{
"type": "COMMIT",
"keyspace": "commerce",
"shard": "0"
}
]
}
{
"events": [
{
"type": "COPY_COMPLETED",
"keyspace": "commerce",
"shard": "0"
},
{
"type": "COPY_COMPLETED"
}
]
}