horaedb icon indicating copy to clipboard operation
horaedb copied to clipboard

Auto refresh route cache

Open ShiKaiWi opened this issue 2 years ago • 1 comments

Describe This Problem

Currently, the route information in the CeresDB instance is configured with a TTL for auto refreshing. And here are some problems of this mechanism:

  • The route entry for a dropped won't expire in time;
  • TTL is hard to be determined: a too short TTL will impose a large route traffic to CeresMeta, and a too long TTL will make invalid route entry exist for a long time;

Proposal

Replace the TTL mechanism with auto refreshing with Shard Version:

  • Let CeresDB instance fetches the versions of its shards from CeresMeta;
  • And fetch the tables belonging to the shard and updating the route entry in the cache;

Additional Context

No response

ShiKaiWi avatar May 12 '23 07:05 ShiKaiWi

The problem is that the route cache used for proxy is involved with all the tables of the whole cluster, so we can't cache routing information of all the tables.

ShiKaiWi avatar May 22 '23 07:05 ShiKaiWi