cloudflow
cloudflow copied to clipboard
Checkpointing configurations are not taking effect for flink streamlet
The below checkpointing configurations are not taking effect when tried to configure enabling the Rocksdb backend with Azure blob storage .
cloudflow.runtimes.flink.config {
flink.state.backend = filesystem
flink.state.checkpoints.dir = "wasbs://<your-container>@$<your-azure-account>.blob.core.windows.net/<object-path>"
flink.fs.azure.account.key.<your-azure-acount>.blob.core.windows.net = "<blob-access-key>"
}
The current workaround is to do it via code:
getContext().env().setStateBackend(new EmbeddedRocksDBStateBackend(true));
getContext().env().getCheckpointConfig().setCheckpointStorage("wasbs://<your-container>@$<your-azure-account>.blob.core.windows.net/<object-path>");
getContext().env().getCheckpointConfig().setCheckpointStorage(new FileSystemCheckpointStorage(file:///tmp/checkpoint-dir));