KAFKA-16717 [3/N]: Add AdminClient.alterShareGroupOffsets

[JimmyWang6]

KAFKA-16717 aims to finish the AlterShareGroupOffsets for ShareGroupCommand part.

[JimmyWang6]

Hi @AndrewJSchofield @smjn @apoorvmittal10 , Sorry for the late response, could you please take a look if there is anything wrong with the current approach? Thanks in advance.

[AndrewJSchofield]

@JimmyWang6 There's a build failure due to spotless check. Please resolve it and merge latest changes.

[chia7712]

@JimmyWang6 could you please fix the build error?

[JimmyWang6]

Hi @chia7712， Thanks for your comment and I have updated the PR, PTAL if you have time.

[JimmyWang6]

Hi @AndrewJSchofield @chia7712 I have updated the PR and resolved several bugs. Here are some of my test scenarios：

1. When a single shareConsumer subscribes to a single topic, consumes some messages, and then exits and , it can subsequently consume all previously consumed messages again after resetting the offset to ealiest.
2. When a single shareConsumer subscribes to mutiple topics, then excute bin/kafka-share-groups.sh --reset-offsets --execute --to-earliest --all-topics --group test-group --bootstrap-server localhost:9092 , the offset of all the topics were reset.
3. When a single shareConsumer subscribes to mutiple topics, consumes some messages, and then exits, it can subsequently consume all previously consumed messages again after resetting the consumption offset.
4. If no topic argument is provided when running bin/kafka-share-groups.sh --bootstrap-server localhost:9092 --group test-group --reset-offsets --dry-run, the following exception occurs: Option [reset-offsets] requires one of these options: [all-topics] or [topic].
5. When none of to-datetime, to-earliest or to-latest are specified, the execution failed with an error Option '[reset-offsets]' requires one of the following scenarios: [[to-datetime], [to-earliest], [to-latest]]
6. For none-exists topic, the execution failed with Some topics none do not exist in share group 'test-group'.

[AndrewJSchofield]

@JimmyWang6 I've spent a while trying out this code and I wasn't able to get the majority of the error scenarios to work as I expected. The problem I think is the Admin.alterShareGroupOffsets method which I don't think was handling top-level errors. I've put in a separate PR https://github.com/apache/kafka/pull/20049 for that which I hope will help things along.

[AndrewJSchofield]

@JimmyWang6 Please merge latest changes. Now that https://github.com/apache/kafka/pull/20049 is merged, the Admin.alterShareGroupOffsets will now return top-level errors correctly. This means kafka-share-groups.sh --reset-offsets in error cases will fail, rather than appearing to work when it did not.

[JimmyWang6]

@AndrewJSchofield So much thanks for your help to resolving the issues. I sincerely apologize for the previous bugs I introduced. I’ll resolve the conflicts and thoroughly test the script as soon as possible to ensure everything works smoothly.

[JimmyWang6]

Hi @chia7712 @AndrewJSchofield , I have updated the PR and addressed the comments, PTAL when get a chance, thanks.

[AndrewJSchofield]

Thanks for the updates. I'm starting to review and test it. Here's a small problem I've found so far. SG1 is an existing share group with 1 member. T2 doesn't exist, so the message about "Not subscribed" is fine. However, the usage message for the tool is printed even though it's a regular expected error.

bin/kafka-share-groups.sh --bootstrap-server localhost:9092 --reset-offsets --group SG1 --execute --topic T2 --to-earliest 
Share group 'SG1' is not subscribed to topic 'T2'
Option                                 Description                           
------                                 -----------                           
--all-groups                           Apply to all share groups.
(( And so on ))

[AndrewJSchofield]

Please fix the checkstyle build error.

[JimmyWang6]

Hi @AndrewJSchofield , I have improved the error handling structure, please take another look when get a chance, thank! Here are my test results:

• Attempt to reset offsets for a consumer group

• Attempt to reset offsets for a non-existent group

• Attempt to reset offsets when there are members in the share group

• Attempt to reset offsets for non-existent topic

• Attempt to reset offsets for existing topic which is not subscribed in the share group

• Successfully to reset offsets to earliest

[AndrewJSchofield]

Still getting the usage message for trying to reset offsets for a consumer group.

bin/kafka-share-groups.sh --bootstrap-server localhost:9092 --reset-offsets --group CG1 --topic T1 --execute --to-earliest
Group CG1 is not a share group.
Option                                 Description                           
------                                 -----------                           
--all-groups                           Apply to all share groups.            
--all-topics                           Consider all topics assigned to a     
                                         share group in the 'reset-offsets'  
                                         process

[AndrewJSchofield]

Please also merge latest changes from trunk. I expect the build will be clean afterwards.

[JimmyWang6]

Still getting the usage message for trying to reset offsets for a consumer group.

bin/kafka-share-groups.sh --bootstrap-server localhost:9092 --reset-offsets --group CG1 --topic T1 --execute --to-earliest
Group CG1 is not a share group.
Option                                 Description                           
------                                 -----------                           
--all-groups                           Apply to all share groups.            
--all-topics                           Consider all topics assigned to a     
                                         share group in the 'reset-offsets'  
                                         process

Oops, I'm sorry that I didn't grasp your meaning, I will try to replace CommandLineUtils.printUsageAndExit() method call by a simple message.

[JimmyWang6]

@AndrewJSchofield I have addressed the below comments as your suggestions. Please take a glance to see if they look good. Thanks! :)

[JimmyWang6]

@AndrewJSchofield I've applied the fixes from both PRs and the long delay issue is resolved. However, there might be conflicts with the current changes cc @TaiJuWu.

[JimmyWang6]

Cheers~