Sub-iterators of ConjunctionDISI are not on the same document

Describe the bug

When performing an intervals query on a field with a custom mapping and analyzer, I get an illegal argument exception with the following reason: Sub-iterators of ConjunctionDISI are not on the same document!. I am not sure if the error is due to an issue with our custom mapping or analyzers, or if it caused by a bug somewhere. Any insight is greatly appreciated.

Related component

Search

To Reproduce

1. Run OpenSearch in a new Docker container

docker run -d -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=password_here" -e 'DISABLE_SECURITY_PLUGIN=true' opensearchproject/opensearch:latest

2. Create a new index with the custom mapping and analyzer

PUT http://localhost:9200/my_index
{
    "mappings": {
        "dynamic_templates": [
            {
                "SearchableFilter": {
                    "match": "*_SearchableFilter",
                    "match_mapping_type": "string",
                    "mapping": {
                        "fields": {
                            "keyword": {
                                "ignore_above": 256,
                                "type": "keyword"
                            }
                        },
                        "search_analyzer": "text_general_search",
                        "type": "text"
                    }
                }
            }
        ]
    },
    "settings": {
        "analysis": {
            "analyzer": {
                "text_general_search": {
                    "filter": [
                        "stop",
                        "lowercase"
                    ],
                    "type": "custom",
                    "tokenizer": "standard"
                }
            }
        }
    }
}

3. Create a new document

POST http://localhost:9200/my_index/_doc
{
    "siblings_SearchableFilter": [
        "a Sister"
    ]
}

4. Perform the following intervals query

POST http://localhost:9200/my_index/_search
{
    "query": {
        "intervals": {
            "siblings_SearchableFilter": {
                "all_of": {
                    "intervals": [
                        {
                            "any_of": {
                                "intervals": [
                                    {
                                        "match": {
                                            "query": "a"
                                        }
                                    },
                                    {
                                        "match": {
                                            "query": "b"
                                        }
                                    }
                                ]
                            }
                        },
                        {
                            "match": {
                                "query": "sister"
                            }
                        }
                    ],
                    "max_gaps": 30,
                    "ordered": true
                }
            }
        }
    }
}

5. Notice you get a 400 response code with the error mentioned above.

Expected behavior

I would expect the intervals query to succeed and return the document created in the reproduction steps.

Additional Details

• The OpenSearch version is the latest since we are running the Docker image opensearchproject/opensearch:latest.
• In my local testing, I ran the image in Docker on WSL running on Windows 11. However, this error also occurs on machines running Ubuntu 22 whose GET response is the following:

{
    "name": "pss-cluster-coordinating-01",
    "cluster_name": "prod_cluster",
    "cluster_uuid": "FFQa8G3RRwqNdct-KMih4g",
    "version": {
        "distribution": "opensearch",
        "number": "2.11.0",
        "build_type": "tar",
        "build_hash": "4dcad6dd1fd45b6bd91f041a041829c8687278fa",
        "build_date": "2023-10-13T02:55:55.511945994Z",
        "build_snapshot": false,
        "lucene_version": "9.7.0",
        "minimum_wire_compatibility_version": "7.10.0",
        "minimum_index_compatibility_version": "7.0.0"
    },
    "tagline": "The OpenSearch Project: https://opensearch.org/"
}

• In my testing, there are a few things that remove the error, each of which also prevent the query from functioning properly and returning the desired document:
  1. Removing max_gaps from the query or setting it to -1.
  2. Removing the custom text_general_search analyzer from the mapping.
  3. Removing either stop or lowercase from the analyzer filter.