chronon icon indicating copy to clipboard operation
chronon copied to clipboard

[Backfill]Remove exception when the keys are all null in unfilledLeftDf

Open yuli-han opened this issue 4 months ago • 1 comments

Summary

When we run the join part backfill on the left, we used to error out when the key are all empty in left table. This doesn't work when we run the join part backfill after bootstrap, because the unfilledLeftDf keys can be all null.

This PR is to remove the exception throw statement in genKeyFilter function. When the keys in unfilledLeftDf are all empty, we will build an empty filter(where keys in ()) in join part and the computeRightTable function will return an empty dataframe, like the case when the bootstrap table covers everything.

Why / Goal

Test Plan

  • [ ] Added Unit Tests
  • [ ] Covered by existing CI
  • [ ] Integration tested

Checklist

  • [ ] Documentation update

Reviewers

@hzding621 @donghanz @pengyu-hou

yuli-han avatar Oct 02 '24 03:10 yuli-han