Is it possible to use Tiledb-MariaDB as a slave of a real MariaDB instance?

[qiuwei]

Is it possible to use Tiledb-MariaDB as a slave of a real MariaDB instance?

We are trying to do this to achieve change data capture of a traditional RDBMS, and do the analytics using tiledb.

[Shelnutt2]

@qiuwei excellent question! TileDB-MariaDB consists of a storage engine (MyTile) which can be compiled along side a normal MariaDB source build[1]. This means you indeed can MyTile running in normal/standard MariaDB instance. This second instance of MariaDB with MyTile installed can be used as a slave to your master/primary instance without a problem.

If you are wanting to use replication to also change storage engines this is not possible, as a general limitation of MariaDB. That is to say if you have a table T1 which is InnoDB on your primary DB, and you setup replication to the secondary MariaDB and you want to make T1 a different storage engine then InnoDB (such as using MyTile) this will not work.

If your scenario is to try to automatically insert from one table to a TileDB array, your best bet is either a trigger or an event. If this is what your looking for I can provide some example setups to achieve this.

We've love to know more about your use case, and I'd be happy to jump on a call to architect out how you can achieve this. Feel free to reach out to us at [email protected].

[1] We also have a traditional mariadb server dockerimage availabl tiledb/tiledb-mariadb-server

[qiuwei]

@Shelnutt2

Thanks for the detailed answer.

Our use case is that we would like to automatically sync data from an OTLP database(say MySQL) to TileDB near realtime, and then do the analytics using TileDB.

I am now considering using solutions like debezium(https://debezium.io/) and write a connector for TileDB. Do you think it is a viable method? Would it cause too many fragments in TileDB? I am more than grateful for any suggestions.