kafka icon indicating copy to clipboard operation
kafka copied to clipboard

KAFKA-17031: Make RLM thread pool configurations public

Open fvaleri opened this issue 1 year ago • 3 comments

According to KIP-950, remote.log.manager.thread.pool.size should be marked as deprecated and replaced by two new configurations: remote.log.manager.copier.thread.pool.size and remote.log.manager.expiration.thread.pool.size.

fvaleri avatar Oct 14 '24 16:10 fvaleri

The default value of -1 means that this will be set to the configured value of remote.log.manager.thread.pool.size, if available; otherwise, it defaults to 10

This is a bit confusing IMO. What if the user sets -2? I think that we can use -1 as default, and fallback when value is <=0. Wdyt? I also have a test to fix.

fvaleri avatar Oct 17 '24 07:10 fvaleri

This is a bit confusing IMO. What if the user sets -2? I think that we can use -1 as default, and fallback when value is <=0. Wdyt? I also have a test to fix.

-2 is invalid value and it will fail. The valid values are -1 and value >0. -1 indicates to be remote.log.manager.thread.pool.size's value if exists else it's default(which is 10)

satishd avatar Oct 17 '24 15:10 satishd

@satishd thanks for the feedback. Latest commit should address your request.

fvaleri avatar Oct 17 '24 16:10 fvaleri

Test failures are unrelated. They work locally and are reported as flaky.

fvaleri avatar Oct 21 '24 15:10 fvaleri

Since this changes the behavior of the new tiered storage configurations, we will need to get it into 3.9 if it is to get in at all. I will commit it now.

cmccabe avatar Oct 21 '24 17:10 cmccabe