TANK icon indicating copy to clipboard operation
TANK copied to clipboard

Topic creation improvements

Open markpapadakis opened this issue 6 years ago • 1 comments

Now, when not running in cluster-aware mode, the directory name for the topic will be prepended with '.', and only when all partitions for that topic have been initialized (in that .directory), that .directory will be atomically renamed to the correct name(i.e the topic name).

This is important because it's possible that partition creation may fail(e.g system fault, FS fault, etc), and we don't want to leave a mess when that happens. That is to say, creating a topic should be an atomic operation.

  • [x] implement cleanup() lambda in Service::process_create_topic()
  • [ ] implement the same idea in Service::reconcile_cluster_topology()

markpapadakis avatar Oct 09 '19 07:10 markpapadakis

Turns out that deleting directories on startup may lead to data loss - because TANK may be misconfigured to use the wrong base directory. Will now generate a warning and will eventually accept a new command-line argument enabling GC on startup.

markpapadakis avatar Oct 10 '19 10:10 markpapadakis