hydroflow
hydroflow copied to clipboard
Checkpoint/migration handling
Split out of #929
Let's get all the state of a transducer exposed for management including checkpoint/restart.
- What about state that persists across executions (e.g. in storage or a replication library)?
- Relates to the "tables" in Bloom being absent from Hydroflow: is state first-class in the IR and separate from ops?
- Might be good to have an explicit data model spec
from @jhellerstein
Arguably our philosophy could be that the Hydroflow runtime itself does not provide intrinsic mechanisms for efficient checkpoint. Instead we defer checkpoint to a system either beneath Hydroflow, or built out of Hydroflow.