wadm icon indicating copy to clipboard operation
wadm copied to clipboard

Refactor wadm state to be less monolithic

Open thomastaylor312 opened this issue 8 months ago • 3 comments

As part of the tradeoffs we made during initial development, we basically decided to store state as a big JSON blob per key. The better way to do this with key value stores is to have a top level key like host_<lattice_id> which contains a list of all keys for host. Each host would have a key like host_<lattice_id>_<host_id> so that it is more efficient to query for a single host. This same pattern also applies for the actor and provider state data we're storing. The current way of storing the data should be refactored to match this better pattern.

One other requirement is that this change should automatically convert the old data style to the new data style so that operators do not have to wipe out the current state store

Please note that this will have knock on effects as the washboard uses the wadm KV bucket right now so once this change is made we need to update the washboard before updating the default wadm version in wash

thomastaylor312 avatar Oct 26 '23 19:10 thomastaylor312