horaedb
horaedb copied to clipboard
Updating schema information for partition table on ceresdb nodes
Describe This Problem
Due to the ability to open partition table on multiple ceresdb nodes, there may be inconsistencies in the schema information maintained in memory. Currently, the approach of cleaning up through errors occurring during writing is relatively rough. Refer to #1244.
Proposal
There are two approaches:
Passive notification:
Currently, the partition table
is also stored in the single-node CeresDB catalog. One idea is to abstract a catalog implementation with TTL capability and store the partitioned table inside it, updating the partition table
periodically.
Drawback: Table alteration operations are not currently frequent. If regular expiration operations for partition table
are carried out due to alter table reasons, it can be costly.
Active notification: Introduce Ceresmeta. After the alter table is completed, notify Ceresmeta, which asynchronously notifies all CeresDB nodes of the schema change. Drawback: Introducing Ceresmeta makes the implementation more complex. Ceresmeta procedure currently does not have persistence functionality, so implementing persistence and retry capabilities is needed.
Additional Context
No response