cockroach
cockroach copied to clipboard
online restore: separate split+scatter from ingest step
This changes online restore to use a two-step link phase, in which the first step is to split and scatter the entire keyspan into which the restore will ingest external files, and then a second step that actually ingests them. Previously online restore did both of these in one step, with individual chunks being split, scattered, and then split further as remote files are ingested into them.
Doing all of the splits and rebalancing before any ingest gives the KV server the most opportunity to rebalance empty ranges, before we ingest data into them that would need to be moved during such rebalancing and increase its cost.
It also makes it easier to determine what step is a bottleneck as we can measure the steps in isolation.
But most importantly, it makes it easier to checkpoint out progress so that we can mark the split-and-scatter phase as completed and then skip it on subsequent retries.
Release note: none. Epic: none.
It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?
:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.
TFTR!
bors r+