kafka icon indicating copy to clipboard operation
kafka copied to clipboard

KAFKA-14747: record discarded FK join subscription responses

Open AyoubOm opened this issue 1 year ago • 1 comments

As described in KAFKA-14747, we are not recording discarded FK join responses in case of receiving a join response for an old record (whose hash value has changed in the left table). This PR adds the record to dropped records sensor

Committer Checklist (excluded from commit message)

  • [ ] Verify design and implementation
  • [ ] Verify test coverage and CI build status
  • [ ] Verify documentation (including upgrade notes)

AyoubOm avatar Feb 20 '24 17:02 AyoubOm

@mjsax Please have a look when you have some time

AyoubOm avatar Feb 26 '24 09:02 AyoubOm

Thanks for the PR! Overall LGTM, but we should add a unit test for the newly added sensor.

Thanks @mjsax for your review. I added tests of the dropped record sensor (I believed we didn't have access to metrics from the context but we have indeed).

Another point, I think there are some missing tests in the foreignkeyjoin package, and problems in naming a test class. For example, ForeignTableJoinProcessorSupplierTest is not testing ForeignTableJoinProcessorSupplier but SubscriptionJoinProcessorSupplier. Hence, I think the ForeignTableJoinProcessorSupplier is not tested.

AyoubOm avatar Mar 04 '24 09:03 AyoubOm

Another point, I think there are some missing tests in the foreignkeyjoin package, and problems in naming a test class. For example, ForeignTableJoinProcessorSupplierTest is not testing ForeignTableJoinProcessorSupplier but SubscriptionJoinProcessorSupplier. Hence, I think the ForeignTableJoinProcessorSupplier is not tested.

Thanks for pointing this out. Would you like to do a follow up PR and rename the test? And maybe also close this test gap (of not, we can also just file a ticket for this for now.)

mjsax avatar Mar 04 '24 23:03 mjsax

Also, thanks for the PR! Merged to trunk.

mjsax avatar Mar 04 '24 23:03 mjsax

Another point, I think there are some missing tests in the foreignkeyjoin package, and problems in naming a test class. For example, ForeignTableJoinProcessorSupplierTest is not testing ForeignTableJoinProcessorSupplier but SubscriptionJoinProcessorSupplier. Hence, I think the ForeignTableJoinProcessorSupplier is not tested.

Thanks for pointing this out. Would you like to do a follow up PR and rename the test? And maybe also close this test gap (of not, we can also just file a ticket for this for now.)

Sure, I will look into that.

AyoubOm avatar Mar 05 '24 07:03 AyoubOm