rocketmq icon indicating copy to clipboard operation
rocketmq copied to clipboard

[Enhancement] Implement Batch Creation of Topics in RocketMQ Admin

Open guyinyou opened this issue 8 months ago • 0 comments

Before Creating the Enhancement Request

  • [X] I have confirmed that this should be classified as an enhancement rather than a bug/feature.

Summary

The RocketMQ project currently supports topic creation through the MQAdmin API. However, the existing API only supports the creation of one topic at a time, which can be inefficient and time-consuming when there is a need to create a large number of topics simultaneously. To address this limitation, I propose an enhancement to the MQAdmin API to include a new UPDATE_AND_CREATE_TOPIC_LIST interface that enables batch creation of topics, significantly improving efficiency for large-scale topic management tasks.

Motivation

The motivation for this enhancement arises from scenarios in distributed systems where multiple topics need to be created for different services or instances. These scenarios are common in large-scale deployments, cloud environments, and multi-tenant platforms where topics are dynamically assigned and managed. The batch creation feature will not only decrease the time required to set up such environments but also minimize the potential for errors during the manual creation process. It aligns with the demands of modern distributed architectures, offering a significant benefit by streamlining the management workflow, reducing operational overhead, and improving user experience. Specific use cases include setting up test environments, migrating or replicating environments, and scaling out services that require multiple topics to be set up in parallel.

Describe the Solution You'd Like

The proposed solution is to introduce a new API interface UPDATE_AND_CREATE_TOPIC_LIST within the MQAdmin toolset. This interface will accept a list of topic configurations and process the batch creation in a single operation. The change will involve enhancing the current administrative command handlers to process multiple topics in one go, leveraging existing infrastructure and ensuring backward compatibility with existing single-topic creation APIs. The implementation will also include appropriate validation checks to ensure atomicity, where all topics need to be successfully created, or none at all, to maintain consistency.

Describe Alternatives You've Considered

none

Additional Context

No response

guyinyou avatar Jun 06 '24 06:06 guyinyou