Implement Kibana/OpenSearch Dashboards dev services

[Javatar81]

This change will add dashboard services to the dev services for extensions/elasticsearch-rest-client and extensions/elasticsearch-java-client. If the distribution is elasticsearch, it will start Kibana as the dashboard. If the distribution is opensearch, it will start OpenSearch-dashboards as the dashboard.

• Fixes #39582

[quarkus-bot[bot]]

/cc @gsmet (elasticsearch), @loicmathieu (elasticsearch), @marko-bekhta (elasticsearch)

[andreaTP]

cc. @yrodiere @holly-cummins

[Javatar81]

Thanks for all your comments. So I would move all my changes into a separate class DevServicesDashboardProcessor with its own properties files and address your proposed changes.

Regarding the Dev Ui. Yes I had that in mind but I planned it as a follow up.

[Javatar81]

I am in the refactoring to move everything related to dashboards into a new processor class named DevServicesElasticsearchDashboardsProcessor. Since both are running in parallel I must make sure that the elasticsearch dev service is already running (but only if the hosts variable is not set). One obvious solution would be to repeat locating the elasticsearch container with a backoff. Is there a more elegant way to wait for the other devservice to be started?

[marko-bekhta]

I am in the refactoring to move everything related to dashboards into a new processor class named DevServicesElasticsearchDashboardsProcessor. Since both are running in parallel I must make sure that the elasticsearch dev service is already running (but only if the hosts variable is not set). One obvious solution would be to repeat locating the elasticsearch container with a backoff. Is there a more elegant way to wait for the other devservice to be started?

oh yes, you could make the ES services produce a build item(s) for you (it would contain a list of hosts) and then you'd use that build items as an input to your dashboards step:

...
startElasticsearchDevService(
    ...
    BuildProducer<ElasticsearchHostBuildItem> elasticsearchHosts
    .. .) {
        ... 

        elasticsearchHosts.produce(...)

    }

and then in the dashboards:

startElasticsearchDashboardsDevService(
    ...
    List<ElasticsearchHostBuildItem> elasticsearchHosts
    .. .) {
        ...
        List<String> eshosts = getAllEsHosts(elasticsearchHosts);
        ..
    }

(maybe you could come up with a better name for ElasticsearchHostBuildItem as well 🙂)

[Javatar81]

All points addressed so far. Please review again.

[quarkus-bot[bot]]

---

:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.

---

Status for workflow Quarkus Documentation CI

This is the status report for running Quarkus Documentation CI on commit b461825c288a143da3ae4567deff1ed13b34b5cd.

:white_check_mark: The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

[!WARNING] There are other workflow runs running, you probably need to wait for their status before merging.

[github-actions[bot]]

🎊 PR Preview f8ca654c8f87d2f7542f2cf86a7eb43ca55119d9 has been successfully built and deployed to https://quarkus-pr-main-51164-preview.surge.sh/version/main/guides/

• Images of blog posts older than 3 months are not available.
• Newsletters older than 3 months are not available.

[quarkus-bot[bot]]

---

:waning_crescent_moon: This workflow status is outdated as a new workflow run has been triggered.

---

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit b461825c288a143da3ae4567deff1ed13b34b5cd.

:white_check_mark: The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.

---

Flaky tests - Develocity

:gear: MicroProfile TCKs Tests

:package: tcks/microprofile-lra

:x: org.eclipse.microprofile.lra.tck.TckRecoveryTests.testCancelWhenParticipantIsUnavailable - History

• Expecting the metric Compensated callback was called Expected: a value equal to or greater than <1> but: <0> was less than <1> - java.lang.AssertionError

java.lang.AssertionError: 
Expecting the metric Compensated callback was called
Expected: a value equal to or greater than <1>
     but: <0> was less than <1>
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at org.eclipse.microprofile.lra.tck.TckRecoveryTests.assertMetricCallbackCalled(TckRecoveryTests.java:210)
	at org.eclipse.microprofile.lra.tck.TckRecoveryTests.testCancelWhenParticipantIsUnavailable(TckRecoveryTests.java:195)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

[Javatar81]

If anything else is needed, please let me know.

[Javatar81]

You can now start your application via -Dquarkus.elasticsearch.hosts=localhost:9200,localhost:9100 -Dquarkus.elasticsearch.devservices.dashboard.enabled=true and if localhost:9200 listens to a Elasticsearch instance and localhost:9100 to a Opensearch instance, the dev service will spin up a Dashboards and a Kibana container connected to the matching backend.

Btw. the same should also work with quarkus.hibernate-search-orm.elasticsearch.hosts when the hibernate-search form-elasticsearch extension has been installed.

[Javatar81]

The dashboards dev services assume that the protocol is http and that there is no security enabled. I think this is sufficient, otherwise we could pass in the quarkus.elasticsearch.protocol, quarkus.elasticsearch.username and quarkus.elasticsearch.password. However, the backend devservice is also not compatible with this, e.g. because it sets xpack.security.enabled = false.

[Javatar81]

I think we need should rename the classes because it is not consistent with the respective technology stacks being used. I propose the following more neutral naming:

• DevServicesElasticsearchDashboardsProcessor-> DevServicesSearchFrontendDashboardsProcessor
• DevservicesElasticsearchConnectionBuildItem -> DevServicesSearchBackendConnectionBuildItem
• DevServicesElasticsearchKibanaProcessor -> DevServicesSearchFrontendKibanaProcessor
• DevservicesElasticsearchDashboardProcessorStrategy -> DevServicesSearchFrontendProcessorStrategy
• DevServicesElasticsearchProcessor -> DevServicesElasticsearchProcessor
• DevservicesElasticsearchProcessorUtils -> DevServicesSearchProcessorUtils
• DevservicesElasticsearchBuildItem -> DevservicesElasticsearchBuildItem
• ElasticsearchCommonBuildTimeConfig -> ElasticsearchCommonBuildTimeConfig
• DevservicesElasticsearchBuildItemsConfiguration -> DevServicesSearchBackendBuildItemsConfiguration
• ElasticsearchRestClientProcessor -> ElasticsearchRestClientProcessor

Btw I found that there is also an inconsistency in camel case naming of DevServices vs. Devservices.

What do you think @marko-bekhta?