kafka icon indicating copy to clipboard operation
kafka copied to clipboard

KAFKA-14546 - Support Partitioner fallback to default

Open jimbogithub opened this issue 2 years ago • 6 comments

Allow custom Partitioners to fallback to default partitioning.

jimbogithub avatar Oct 11 '23 20:10 jimbogithub

See https://issues.apache.org/jira/browse/KAFKA-14546 for rationale.

jimbogithub avatar Oct 11 '23 20:10 jimbogithub

This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch)

If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.

github-actions[bot] avatar Jan 10 '24 03:01 github-actions[bot]

This PR is still valid and desired, has no merge conflicts and does build despite Jenkins protestations. I do not have the ability to add Reviewers.

jimbogithub avatar Jan 10 '24 03:01 jimbogithub

@jimbogithub - thanks for the PR, i've got a few questions to clarify:

  • KafkaProducer.partition(...) not throw IllegalArgumentException if the Partitioner returns RecordMetadata.UNKNOWN_PARTITION : I'm not sure if any partitioner actually return this UNKNOWN_PARTITION. it should be returned by the producer.partition no?
  • to the question above, I think different partitions has different implementations of handling missing topics. but since they all return some partition, are the default behaviors insufficient? bottom line is I'm not 100% convincing that the fallback to default is actually needed.
  • there's no test for the change you made. i think it would be good to add some to demonstrate some of the use cases.
  • I would also update the PR description. you can just use whatever described in the PR.

@jolshan - if you get time maybe you can review this? it seems like you implemented a few partitioner so you are probably the better person to speak about this... thanks!

philipnee avatar Jan 10 '24 05:01 philipnee

I think this change would require a KIP since it modifies the public API.

jolshan avatar Jan 10 '24 16:01 jolshan

This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch)

If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.

github-actions[bot] avatar Jun 27 '24 03:06 github-actions[bot]

This PR has been closed since it has not had any activity in 120 days. If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open the PR and ask for a review.

github-actions[bot] avatar Jan 03 '25 03:01 github-actions[bot]