[BUG] Plain highlighter fails with neural query and filter

[tenpura-shrimp]

Describe the bug

Passing the filter parameter causes the search query to fail when the plain highlighter is enabled. I did a query with the filter parameter following the docs here: https://opensearch.org/docs/latest/query-dsl/specialized/neural/

The failure happens even when the filter is match_all, which shouldn't affect the results, so this appears to be a bug in the opensearch implementation.

Related component

Search

To Reproduce

Run the following query

GET /utterance-3/_search
{
  "highlight" : {
    "fields" : {
      "plainText" : {
        "type" : "plain"
      }
    },
    "fragment_size" : 10000,
    "post_tags" : [ "bbq" ],
    "pre_tags" : [ "wtf" ]
  },
  "query" : {
    "neural" : {
      "passage_embedding" : {
        "query_text" : "exam",
        "model_id" : "ubssVI4BA1L0FKVMp3Mf",
        "k" : 10,
        "filter" : {
          "match_all": {}
        }
      }
    }
  },
  "size" : 21
}

The result is the following failure:

{
  "took": 314,
  "timed_out": false,
  "_shards": {
    "total": 6,
    "successful": 3,
    "skipped": 0,
    "failed": 3,
    "failures": [
      {
        "shard": 0,
        "index": "utterance-3",
        "node": "bawq-EOJSFWPoEJNao2LUg",
        "reason": {
          "type": "unsupported_operation_exception",
          "reason": null
        }
      }
    ]
  },
  "hits": {
    "total": {
      "value": 60,
      "relation": "eq"
    },
    "max_score": 0.6048681,
    "hits": []
  }
}

Removing the filter makes it work

Expected behavior

The query should succeed

Additional Details

Plugins Please list all plugins currently enabled.

Screenshots If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: Amazon Opensearch Service
• Version 2.11.1

Additional context Add any other context about the problem here.

[dblock]

Thanks for reporting this. Try turning it into a REST test?

[andrross]

@opensearch-project/triage Can this be transferred to https://github.com/opensearch-project/neural-search as this the neural query is executed by that plugin?

[navneet1v]

@tenpura-shrimp can you please add the index mappings which you used to create the index.

[jmazanec15]

Neural query does not work with highlighting this is being tracked in #145 . Closing this one