venice
venice copied to clipboard
[controller] Enable backend to do dual reads on DVC incremental push status
Summary
Controller will first try to read from version level key for DVC incremental push status; if version level key exists and the overall status is END_OF_INCREMENTAL_PUSH, controller would still check the partition level key and combine the result from both keys, in case the DVC is partially upgraded.
Other changes: Add a few more logs for each job status polling request to help us further confirm that the version level keys are working well, besides the signal of successful push jobs.
How was this PR tested?
New tests in progress
Does this PR introduce any user-facing changes?
- [x] No. You can skip the rest of this section.
- [ ] Yes. Make sure to explain your proposed changes and call out the behavior change.