mypipe
mypipe copied to clipboard
mardambey
MySQL binary log consumer with the ability to act on changed rows and publish changes to different systems with emphasis on Apache Kafka.
ColumnType.scala gives us some errors on VAR_STRING and STRING values. Data stored in these kinds of columns seems to be typically byte[]. Implicit conversion using `column.valueOption[String].getOrElse("")` fails. We now use...
Hi, I'm trying to snapshot a large table (~100 million rows) to kafka to bootstrap a replica of a mysql table on HDFS. I'm using the `--no-transaction` flag because I...
i have a mysql server on my remote server with row binlog enabled,then i want to implement a data collection bus using kafka(whether source is mysql or other format log...
Hi is it possible to publish to maven for easier use, at least mypipe-api? Thanks.
Hi This may or may not be regarded as a valid issue for you, but I've tried building a very similar solution to this in the past and this is...
How does mypipe recover after a certain node in a cluster dies? mysql-binlog-connector-java 0.2.1 now has support for GTIDs. What are the steps to integrate this? Support for COM_BINLOG_DUMP_GTID (`BinaryLogClient`::[gtidSet](https://github.com/shyiko/mysql-binlog-connector-java/commit/3f30768791ef61ab0a83bd2bdb98af80bc799abd#diff-7addc4e3eed6e9254e088abc015ac8adR229))...
It looks like CacheableTableMapBehaviour doesn't properly deal with tableMap events. We have a table Foo that was mapped to id 71 for a while. We then had a failover, and...
It would be nice, if we can have automatic avro schemas generated for mypipe tables.
Hi. My issue is quite similar to https://github.com/mardambey/mypipe/issues/15 For all kinds of query being made on the database, I am only able to receive empty sets of mutations. ``` UPDATE...
