kafka-tutorials icon indicating copy to clipboard operation
kafka-tutorials copied to clipboard

reorder stream and table inserts in fleet mgmt recipe

Open davetroiano opened this issue 2 years ago • 1 comments

  • this mitigates null values from table side of join

Description

https://github.com/confluentinc/kafka-tutorials/issues/1286

Staging Docs

These links likely won't be live since semaphore build currently disabled:

http://kafka-tutorials-staging.s3-website-us-west-2.amazonaws.com/fleet-mgmt-fix-null-table-side-values/ http://kafka-tutorials-staging.s3-website-us-west-2.amazonaws.com/fleet-mgmt-fix-null-table-side-values/fleet-management/confluent.html

Existing tutorial checklist

  • [x] Validate hyperlinks
  • [x] Spell check (e.g. using aspell)
  • [x] Full code validated in Confluent Cloud
  • [x] SQL style/syntax consistent to existing recipes
  • [x] Validate presentation with bundle exec jekyll serve --livereload

davetroiano avatar Jun 16 '22 20:06 davetroiano

I'm opening this as a mitigation but I don't believe it's a true fix. AFAIK the true fix would require window / grace period on stream-table joins. That being said, I've observed it to be effective -- without it, I observed null table-side values on 3 attempts in Confluent Cloud and in 5 local docker runs. With it, I don't get nulls on 3 attempts in Confluent Cloud nor in 5 local docker runs.

davetroiano avatar Jun 16 '22 20:06 davetroiano