temporal
temporal copied to clipboard
temporalio
Add support for Yugabyte YCQL backend
We currently leverage both Yugabyte YCQL and Temporal and would love to see native support for Temporal to use the YCQL variant of its Cassandra driver, so we do not need to maintain both a YB and Cassandra cluster.
I should add: I was able to bring Temporal up on YB's YSQL using Temporal's postgres driver, but the performance observed when running Maru was too low to be considered. This may be something else to sort out.
Can you connect Temporal to YB using Cassandra driver? YB protocol is Cassandra compatible, it should work. Maybe after some fixes as we did to support Scylla DB.
I did try this, but there are a few things in the Temporal Cassandra driver that are not compatible with YB.
One of them was that you need to enable transactions to create secondary indices with with transactions = { 'enabled' : true }. Another was something to do with the use of TTL expirations, IIRC.
My guess is it's a very minor modification to go from the Temporal Cassandra driver to a YB-compatible one, but I didn't look too deep.
Secondary indexes are mostly for visibility database. And I believed we removed TTL usage everywhere from the code.
So yes, it should be relatively easy, especially for someone with YB background. Contributions are welcome!
Manetu may be interested in sponsoring/tackling this. Is there any guidance as far as how best to develop/propose a PR of this nature that has a shot of being merged upstream?
@ghaskins, I see you had bad performance when testing with YugabyteDB YSQL. YugabyteDB's default configuration may not be appropriate. For example, all indexes are hash-sharded, which is not helpful for range queries. Here is a docker compose with parameters that should behave like PostgreSQL https://github.com/FranckPachot/temporalio-docker-compose/blob/main/docker-compose-yugabytedb.yml
Do you have a way to test performance?
