kafka icon indicating copy to clipboard operation
kafka copied to clipboard

DRAFT KAFKA-18692: Consider to unify KStreamImpl "repartitionRequired" with GraphNode "keyChangingOperation"

Open appchemist opened this issue 9 months ago • 8 comments

That PR is a POC.

For now, I only focused on modifying KStreamImpl.repartitionRequired to replace it with GraphNode.keyChangingOperation.

So I'll be working on it further to make sure it's the right fix and need to write some test code.

Committer Checklist (excluded from commit message)

  • [ ] Verify design and implementation
  • [ ] Verify test coverage and CI build status
  • [ ] Verify documentation (including upgrade notes)

appchemist avatar Feb 04 '25 14:02 appchemist

Hi, @mjsax If you have a moment, Please take a look

appchemist avatar Feb 04 '25 14:02 appchemist

Thanks for this draft -- might take a few days until I find time to take a look.

mjsax avatar Feb 05 '25 02:02 mjsax

Hi, @mjsax After reuse InternalStreamsBuilder#getKeyChangingParentNode(). Some Tests are failed. So, I'll looking for these failed tests I haven't had much time lately, so I haven't looked at it much.

After Analyzing the cause, I'll request the review again

However, I would appreciate it if you could take a look at the following comment when you have time. https://github.com/apache/kafka/pull/18800#discussion_r1941275162

appchemist avatar Feb 17 '25 12:02 appchemist

:streams:testAll now succeed.

@ mjsax If you have a moment, Please take a look.

If the POC is worth applying, I'll resolve the conflict.

appchemist avatar Feb 22 '25 06:02 appchemist

@mjsax kindly ping

appchemist avatar Mar 05 '25 13:03 appchemist

@appchemist -- Sorry for dropping the ball on this work. -- I would have time now, to support you with with PR, if you are still interested.

mjsax avatar Jun 11 '25 21:06 mjsax

@mjsax Of course, I'm still interested. Thank you for your time.

appchemist avatar Jun 12 '25 06:06 appchemist

I've resolve the conflict. ./gradlew clean :streams:testAll is all passed

appchemist avatar Jun 15 '25 10:06 appchemist

Since the existing commit had conflicts when rebasing onto the latest trunk, I created a new commit instead.

All streams tests pass, but some integration tests occasionally fail.

appchemist avatar Sep 07 '25 15:09 appchemist