pulsar-manager
pulsar-manager copied to clipboard
Create Index on topics_stats to fix tenants tab latency
Fixes #419
(or if this PR is one task of a github issue, please add Master Issue: #<xyz> to link to the master issue.)
Master Issue: #
Motivation
Tenants tab in Pulsar Manager is slow and takes forever to load sometime.
Modifications
We are using Postgres for data storage and noticed that while loading tenants page it is stuck at one query and takes forever to load. Tenants page request load data from topics_stats table and load only one topic status with latest timestamp. The query used here https://github.com/sourabhaggrawal/pulsar-manager/blob/5343f09f381729ef6798ddf1bffccd55aaff04b8/src/main/java/com/manager/pulsar/mapper/TopicsStatsMapper.java#L37 fetch only single record by ordering on the timestamp column everytime, with too many records this query takes too much time .
Added the index to get result faster from the query.
Verifying this change
- [x] Make sure that the change passes the
./gradlew buildchecks.
Hello @tuteng Please take a look.
Can you handle the ci exception? @sourabhaggrawal
Hi @tuteng the change is in sql file to create an index and should not impact any existing test case. Below is the test case which is failing org.apache.pulsar.manager.service.TopicsServiceImplTest > getTopicsStatsImplTest FAILED java.lang.AssertionError at TopicsServiceImplTest.java:125
can you please help me understand how can I fix this or rerun the build?
I don't seem to be able to re-run the build either, I think you could do an empty commit and push to this branch again and that might re-trigger the build. @sourabhaggrawal
git commit --allow-empty
git push origin sourabhaggrawal:master
Hi @tuteng Thanks for the suggestion and looks like the build is triggered. I see 1 workflow is waiting for an approval. Can you please help me get it resolved too ?
@tuteng @eolivelli Can you please approve the test workflow run?
Hello @eolivelli @tuteng, please approve the test workflow run.