Altinity
[WIP] Feature/add update delete support
Hello @ygnuss
Thanks for the amazing work! Is there anything that I can help with to expedite this PR?
Is there a way we can use this feature ?
Currently we run ETL scripts at every 5mins to fetch data from OLTP(say MySQL) and put it to Clickhouse (Collapsing Merge Tree). Still by running ETLs we are not be able to match the realtimeness and the handling updates is not that easy in Clickhouse. Is there a good you would suggest ?
Is there a way I can help in this feature addition, this is much needed.
Hi!!
We developed this feature in Tinybird and tried it with one of our use cases. Unfortunately, the performance was not the expected. Clickhouse, since it is designed for inserts not for update and deletes, does not perform well with this changes. We tried it with one of our use cases and after some time (weeks) the performance significantly degraded and finally blocked. Clickhouse was not able to apply the mutations fast enough.
However, we have worked on other alternatives and we are currently able to handle this kind usage patterns (CDC) in real time. We already have it in production with several customers. You can try out Tinybird for free. You can check more information in our guides and our documentation documentation.
Let me know if you need more help!