[BUG]Cross cluster replication API doesn't fail if multiple pattern is configured in same rule.

[chaitrahegde115]

What is the bug? Cross cluster replication API doesn't fail if multiple pattern is configured in same rule.

How can one reproduce the bug? I am using Opensearch 2.7.0 version. I created a new replication rule which contains multiple patterns to be replicated like below. But curl API doesn't throw any error and it doesn't replicate any data from leader to follower. Steps:

1. Create a replication rule with multiple index pattern

curl -XPOST -k -H 'Content-Type: application/json' -u 'ccr-user:ccruser' 'https://localhost:9200/_plugins/_replication/_autofollow?pretty' -d '
{
   "leader_alias" : "leader-cluster",
   "name": "replication-rules",
   "pattern": "log*, test*",
   "use_roles":{
      "leader_cluster_role": "cross_cluster_replication_leader_full_access",
      "follower_cluster_role": "cross_cluster_replication_follower_full_access"
   }
}'
{
  "acknowledged" : true
}

2. If I check /autofollow_stats it lists both indexpatterns. But it doesn't replicate any data.

curl -XGET -u 'admin:admin' -k 'https://localhost:9200/_plugins/_replication/autofollow_stats' {
  "num_success_start_replication" : 0,
  "num_failed_start_replication" : 0,
  "num_failed_leader_calls" : 0,
  "failed_indices" : [ ],
  "autofollow_stats" : [
    {
      "name" : "replication-rules",
      "pattern" : "log*, test*",
      "num_success_start_replication" : 0,
      "num_failed_start_replication" : 0,
      "num_failed_leader_calls" : 0,
      "failed_indices" : [ ],
      "last_execution_time" : 1688106988643
    }
  ]
}

What is the expected behavior? Atleast curl command should fail if multiple index pattern replication can't be configured and not supported.

What is your host/environment?

• OS: k8s 1.21(Linux)
• Version 1.21
• Plugins: cross cluster replication

[monusingh-1]

Hi @chaitrahegde115, you can only have 1 autofollow pattern in a autofollow rule. To enable autofollow on 2 patterns, create two autofollow rules like below:

curl -XPOST -k -H 'Content-Type: application/json' -u 'ccr-user:ccruser' 'https://localhost:9200/_plugins/_replication/_autofollow?pretty' -d '
{
   "leader_alias" : "leader-cluster",
   "name": "replication-log",
   "pattern": "log*",
   "use_roles":{
      "leader_cluster_role": "cross_cluster_replication_leader_full_access",
      "follower_cluster_role": "cross_cluster_replication_follower_full_access"
   }
}'


curl -XPOST -k -H 'Content-Type: application/json' -u 'ccr-user:ccruser' 'https://localhost:9200/_plugins/_replication/_autofollow?pretty' -d '
{
   "leader_alias" : "leader-cluster",
   "name": "replication-test",
   "pattern": "test*",
   "use_roles":{
      "leader_cluster_role": "cross_cluster_replication_leader_full_access",
      "follower_cluster_role": "cross_cluster_replication_follower_full_access"
   }
}'

In the case where you put "pattern": "log*, test*", the string will be considered as a single autofollow pattern, since there is space " " in the pattern and Opensearch index name cannot have " ", the replication rule will not match any index.

[chaitrahegde115]

Hi, Creating multiple replication rule is fine. But curl command should not give "acknowledged" : true as this would be misleading. If multiple patterns are configured and since it's not supported, the curl output should be some error.

[skumarp7]

Hi,

Request feedback from maintainers on whether the requested change is valid/desirable from the plugin's design, and if yes, I would like to contribute for this.

[skumarp7]

Hi ,

Since there is no validation done to check if the pattern configured intthe autofollow API is a valid indexpattern, I would like to raise a PR to add validations to check if it's a valid indexpattern thats configured in the autofollow pattern by checking for these invalid characters [ "\" , "/" , "?" , '"' , "<", ">", "|", " ", "," , "#" , ":" ]. Request feedback from maintainers on whether the changes are valid.