[FEATURE] Support plugins in client

[dsinghal-nice]

What is the bug?

OpenSearchClient is not having any way to use _plugins/_sql?format=json API.

How can one reproduce the bug?

POST _plugins/_sql?format=json { "query": "SELECT * FROM books_test WHERE year > 1950" }

Trying to execute such SQL based query using OpenSearchClient, but there's no way to do it.

[dblock]

We are working on adding plugins via a new code generator (first parts in https://github.com/opensearch-project/opensearch-java/pull/1052) from spec (https://github.com/opensearch-project/opensearch-api-specification). If you would like to help, a good place to start is to contribute the missing specs via https://github.com/opensearch-project/opensearch-api-specification/issues/235. There are some PRs in progress, e.g. https://github.com/opensearch-project/opensearch-api-specification/pull/456. @Xtansia is working on the generator parts.

[dblock]

Btw, you can use the generic client interface that should work for SQL, https://github.com/opensearch-project/opensearch-java/blob/main/guides/generic.md.

[dsinghal-nice]

Thanks @dblock for the information, I will try to help for missing specs.

Just one question, When I use generic client to perform a SQL query search, I get the response Body in HitsMetadata format, is there a way to convert the Body to POJO, I checked the generic.md file as well, but the HitsMetadata class does not have _DESERIALIZER. Can you tell me if there's a way already available to convert below response in POJO or I need to write my own logic?

{
    "took": 25,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 0.0,
        "hits": [{
                "_index": "books_test",
                "_id": "2",
                "_score": 0.0,
                "_source": {
                    "id": "2",
                    "title": "Kafka on the shore Revised",
                    "year": 1963,
                    "author": "Philip Gabriel, Haruki Murakami"
                }
            }
        ]
    }
}

I tried below approach as mentioned in generic.md but the _DESERIALIZER is not present.

final CreateIndexResponse r = response.getBody()
      .map(b -> Bodies.json(b, CreateIndexResponse._DESERIALIZER, jsonpMapper))
      .orElse(null);

[dblock]

I am not sure. Generally .generic() and non-generic don't mix, but maybe @reta or @Xtansia know how?

[reta]

We have the issue regarding that: https://github.com/opensearch-project/opensearch-java/issues/540

[reta]

I am not sure. Generally .generic() and non-generic don't mix, but maybe @reta or @Xtansia know how?

final CreateIndexResponse r = response.getBody()
      .map(b -> Bodies.json(b, CreateIndexResponse._DESERIALIZER, jsonpMapper))
      .orElse(null);

This approach should work when POJO is properly specified

[dsinghal-nice]

I am not sure. Generally .generic() and non-generic don't mix, but maybe @reta or @Xtansia know how?

final CreateIndexResponse r = response.getBody()
      .map(b -> Bodies.json(b, CreateIndexResponse._DESERIALIZER, jsonpMapper))
      .orElse(null);

This approach should work when POJO is properly specified

@reta I am trying to run /plugins/_sql, which returns response in HitsMetadata format (I couldn't find any other class that matches the response of _sql API), the HitsMetadata does not have a _DESERIALIZER, so I can't do Bodies.json thing

[reta]

@reta I am trying to run /plugins/_sql, which returns response in HitsMetadata format (I couldn't find any other class that matches the response of _sql API), the HitsMetadata does not have a _DESERIALIZER, so I can't do Bodies.json thing

Correct, as per https://github.com/opensearch-project/opensearch-java/issues/540, the plugin specific clients are not yet available, so the _DESERIALIZER (or equivalents) need to be coded manually at the moment.

[Xtansia]

I am trying to run /plugins/_sql, which returns response in HitsMetadata format (I couldn't find any other class that matches the response of _sql API), the HitsMetadata does not have a _DESERIALIZER, so I can't do Bodies.json thing

@dsinghal-nice It looks more like it returns the equivalent of SearchResponse, HitsMetadata is what's nested under hits. Both expose a createSearchResponseDeserializer or createHitsMetadataDeserializer respectively which is what you'd use instead of _DESERIALIZER as you need to provide the document type deserializer.

[dsinghal-nice]

Hi @Xtansia, @reta, @dblock

Thanks for the help !!

Can you please help me with below questions as well ?

1. While creating connection pool if the connection fails then does the library retries and how to configure in this case ?
2. If connection lost while performing some operation like search or bulk insert, then how can I retry to initiate the connection back, does the library retries by itself ?
3. How to create connection pooling from Java client in opensearch or how can I check how many active connections are live from my client ?
4. How many concurrent request does the client supports ?