kafka icon indicating copy to clipboard operation
kafka copied to clipboard

KAFKA-13022: Optimize ClientQuotasImage#describe

Open FrankYang0529 opened this issue 10 months ago • 4 comments

In previous implementation, the ClientQuotasImage#describe goes through all ClientQuotaEntity and checks whether entity type and entity name are matched the filter. In this PR, it goes through all ClientQuotaEntity#entries in the constructor and build a new data structure Map<String, Map<String, Set<Entry<ClientQuotaEntity, ClientQuotaImage>>>>. The first layer is entity type. There are only three entity types: user, client-id, and ip. The second layer is entity name. The last layer is a set of matched entries. With this data structure, we can speed up describe function.

Correctness is covered by ClientQuotasRequestTest.

Benchmark result:

  • trunk
    Benchmark
    

(eachEntityCount) Mode Cnt Score Error Units ClientQuotasImageDescribeBenchmark.describeDefault 10 avgt 2 422.020 ns/op ClientQuotasImageDescribeBenchmark.describeDefault 100 avgt 2 4380.829 ns/op ClientQuotasImageDescribeBenchmark.describeDefault 1000 avgt 2 54567.929 ns/op ClientQuotasImageDescribeBenchmark.describeExact 10 avgt 2 461.441 ns/op ClientQuotasImageDescribeBenchmark.describeExact 100 avgt 2 4620.061 ns/op ClientQuotasImageDescribeBenchmark.describeExact 1000 avgt 2 55275.026 ns/op ClientQuotasImageDescribeBenchmark.describeSpecified 10 avgt 2 651.988 ns/op ClientQuotasImageDescribeBenchmark.describeSpecified 100 avgt 2 5477.147 ns/op ClientQuotasImageDescribeBenchmark.describeSpecified 1000 avgt 2 60935.943 ns/op


* branch

Benchmark (eachEntityCount) Mode Cnt Score Error Units ClientQuotasImageDescribeBenchmark.describeDefault 10 avgt 2 10.666 ns/op ClientQuotasImageDescribeBenchmark.describeDefault 100 avgt 2 11.317 ns/op ClientQuotasImageDescribeBenchmark.describeDefault 1000 avgt 2 11.409 ns/op ClientQuotasImageDescribeBenchmark.describeExact 10 avgt 2 68.551 ns/op ClientQuotasImageDescribeBenchmark.describeExact 100 avgt 2 61.202 ns/op ClientQuotasImageDescribeBenchmark.describeExact 1000 avgt 2 65.592 ns/op ClientQuotasImageDescribeBenchmark.describeSpecified 10 avgt 2 366.921 ns/op ClientQuotasImageDescribeBenchmark.describeSpecified 100 avgt 2 4135.522 ns/op ClientQuotasImageDescribeBenchmark.describeSpecified 1000 avgt 2 52720.811 ns/op

FrankYang0529 avatar Mar 03 '25 12:03 FrankYang0529

This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please leave a comment asking for a review. If the PR has merge conflicts, update it with the latest from the base branch.

If you are having difficulty finding a reviewer, please reach out on the [mailing list](https://kafka.apache.org/contact).

If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.

github-actions[bot] avatar Jun 20 '25 03:06 github-actions[bot]

@FrankYang0529 Thanks for looking into this. Can you rebase on trunk? I agree with @ahuang98, a few comments could help.

mimaison avatar Sep 29 '25 14:09 mimaison

Sorry for the late reply. I will update the PR this week.

FrankYang0529 avatar Sep 29 '25 14:09 FrankYang0529

Thanks for the PR, the improvement is impressive!

That said the code is kind of hard to read. I wonder if we should add unit tests for the describe() method?

mimaison avatar Dec 04 '25 16:12 mimaison