horaedb icon indicating copy to clipboard operation
horaedb copied to clipboard

Updating schema information for partition table on ceresdb nodes

Open chunshao90 opened this issue 1 year ago • 0 comments

Describe This Problem

Due to the ability to open partition table on multiple ceresdb nodes, there may be inconsistencies in the schema information maintained in memory. Currently, the approach of cleaning up through errors occurring during writing is relatively rough. Refer to #1244.

Proposal

There are two approaches:

Passive notification: Currently, the partition table is also stored in the single-node CeresDB catalog. One idea is to abstract a catalog implementation with TTL capability and store the partitioned table inside it, updating the partition table periodically. Drawback: Table alteration operations are not currently frequent. If regular expiration operations for partition table are carried out due to alter table reasons, it can be costly.

Active notification: Introduce Ceresmeta. After the alter table is completed, notify Ceresmeta, which asynchronously notifies all CeresDB nodes of the schema change. Drawback: Introducing Ceresmeta makes the implementation more complex. Ceresmeta procedure currently does not have persistence functionality, so implementing persistence and retry capabilities is needed.

Additional Context

No response

chunshao90 avatar Oct 09 '23 07:10 chunshao90