[improve][pip] PIP-449: Improve metadata sync to have exclusions in syncing specific config matching regex pattern across onprem and cloud clusters

[vraulji567]

Motivation

The pulsar metadata sync does not have a way to exclude specific config from syncing across clusters. For example, a cluster need to have a separate config in on-prem and cloud for it to function correctly for geo-replication. Syncing the cluster config creates issues in geo-replication. Enhance pulsar metadata synchronizer to exclude cluster config from syncing across clusters. Apply the exclusion in a way that a specific destination cluster can ignore a config sync based on the pattern set at the destination cluster level. Pulsar metadata synchronizer will publish the metadata change event to the destination cluster, which may or may not exclude the event depending on the exclusion pattern set in the destination broker config. Multiple exclusions patterns can be configured and sync would be excluded if any of the patterns match.

Modifications

• Add config metadataSyncEventExclusions to exclude sync. Provide regex pattern eg /admin/clusters/.*, /admin/(?!.test). to exclude the config sync event.
• Enhance PulsarMetadataEventSynchronizer to exclude syncing events configured with metadataSyncEventExclusions

Verifying this change

• [ ] Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

• Added tests to verify different regex patterns to apply exclusions to the metadata sync.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• [ ] Dependencies (add or upgrade a dependency)
• [ ] The public API
• [ ] The schema
• [ ] The default values of configurations
• [ ] The threading model
• [ ] The binary protocol
• [ ] The REST endpoints
• [ ] The admin CLI options
• [ ] The metrics
• [ ] Anything that affects deployment

Documentation

• [ ] doc
• [ ] doc-required
• [x] doc-not-needed
• [ ] doc-complete

[github-actions[bot]]

@vraulji567 Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

[codelipenghui]

Hi @vraulji567 Thanks for contributing this feature. Since this feature is going to add new features to the metadata sync, could you please help create a proposal following https://github.com/apache/pulsar/tree/master/pip#readme

[vraulji567]

Hi @vraulji567 Thanks for contributing this feature. Since this feature is going to add new features to the metadata sync, could you please help create a proposal following https://github.com/apache/pulsar/tree/master/pip#readme

I have created the proposal for the change.