Piotr Grabowski
Piotr Grabowski
Add support for including frozen collections in generated Kafka changes: - frozen `MAP` - frozen `LIST` - frozen `SET` - frozen UDTs - tuples (at the moment, all tuples are...
Read windows with present lastConsumedChangeId more efficiently. Instead of filtering out rows on the client side, perform two queries to Scylla: first - reading a single partial stream, second -...
Changes the strategy the CDC library uses to generate windows to read the CDC log. Previously, when a library was started, it would read queryTimeWindowSize large windows starting from |now...
When there is an exception while reading a window, we restart it with `TaskState` pointing to last correctly read change. After restarting, we read the entire window from beginning, but...
As observed in a CI run (but very hard to reproduce, this run somehow reused a CI container from a previous run which might have polluted the environment): https://github.com/scylladb/scylla-cdc-java/runs/3477278901 The...
When the library starts up reading from the start of generation, there are no delays/throttling of reading the old windows.
Currently, the library handles schema change incorrectly. Moreover, `Driver3SchemaBuilder` could be prone to race condition issues: https://github.com/scylladb/scylla-cdc-java/blob/cf6a7090ac18f5e43da72ee325413940d016b11e/scylla-cdc-driver3/src/main/java/com/scylladb/cdc/cql/driver3/Driver3SchemaBuilder.java#L74 as it is not specified that this read base table schema will be...
In this commit: https://github.com/scylladb/scylla-cdc-java/commit/77abc2358678809d08b3084aafc57d49cd6559df, `getUnsafeBytes` was introduced back. This API could make the replicator source code easier, eliminating the need to translate between different representations. Additionally, the replicator now manually...
The `POST_IMAGE` mode of replicator when it encounters a `INSERT` or `UPDATE` CDC log row, it "suspends" replication of such row, instead waiting for `POST_IMAGE` row. When it finds a...
In the upcoming version of ScyllaDB there's a new architecture based on tablets: https://github.com/scylladb/scylladb/pull/13387 The Rust Driver should be updated to support load balancing, reading of metadata of keyspaces with...