chronon icon indicating copy to clipboard operation
chronon copied to clipboard

Make sure there is a mutation table for mutation sources

Open pengyu-hou opened this issue 9 months ago • 0 comments

Summary

In this PR, we will check if there is a mutation table in the entity source to determine if it is a mutation source. The entity source itself is not enough.

Before this PR: In the Entity JoinSource case, it will try to generate the mutation data frame. However, there are no mutation related fields like is_before or mutation_time_column. Therefore, the job will exit early with some schema could not resolved error.

Why / Goal

Test Plan

  • [ ] Added Unit Tests
  • [ ] Covered by existing CI
  • [ ] Integration tested
  • [x] tested with Airbnb internal case

Checklist

  • [ ] Documentation update

Reviewers

@airbnb/zipline-maintainers @nikhilsimha @hanyuli1995 @donghanz @ezvz

pengyu-hou avatar May 23 '24 21:05 pengyu-hou