Graylog2
Delete index via API not scaling with huge amount of index sets
Expected Behavior
Fast deleting of indices via Graylog REST API independent of index set amount
Current Behavior
Deleting of indices gets slower as more index sets the cluster has. The reason can be found here, as graylog iterates over all index sets if an index gets deleted by api: https://github.com/Graylog2/graylog2-server/blob/5.2/graylog2-server/src/main/java/org/graylog2/indexer/MongoIndexSetRegistry.java#L234
This Method iterates over all index Sets which are currently in the cluster to check if the current write index equals the index which should be deleted. In our env this are over 500 Calls to Opensearch (/_alias/<index_prefix>_deflector) As this calls are made one after another the response time for the delete calls stack ups to ~5-10 Minutes.
Possible Solution
A much nicer approach would be to test /<index_name>/_alias if this response contains any alias ending with _deflector
(All methods in this Class seam not to scale very well with rising index sets, we observed problems with deleting sofar. )
Steps to Reproduce (for bugs)
- Create high amound index sets
- try to delete an index in graylog via rest api
- wait very long
Context
There are various reasons why one would like to delete an index via graylog webui or by rest api.
Your Environment
- Graylog Version: 5.2.4
- Java Version: JDK 17
- OpenSearch Version: 2.12
- MongoDB Version: 5
- Operating System: RHEL 8
- Browser version: Firefox latest ESR
Hey @HenryTheSir!
Thank you very much for reporting this and providing very helpful details!
