Separate OpenSearch Dashboards from Opensearch for vital functions

[sastorsl]

Is your feature request related to a problem? Please describe.

It is problematic that OpenSearch Dashboards ("the client") must have OpenSearch ("the backend") up and ready to function.

1. An intermittent issue with the backend causes the client to not give proper feedback to the client.
2. A "lifecycle" probe, i.e. on k8s can't detect if the client is working unless the backend is up
3. On startup the end user depends on both the client and the backend to get any / valuable feedback.

Describe the solution you'd like

1. The client should be up and continue to connect to the backend
2. An API endpoint should be provided to detect that OpenSearch Dashboards is responding
3. A second API endpoint should be provided to detect if the whole chain - including the backend - is operational
4. The client should provide feedback to end uses - "The OpenSearch backend database is unavailable at the moment..." or something similar

Describe alternatives you've considered

N/A

OpenSearch Dashboards returns 50x now on all endpoints until OpenSearch (backend) is responding.

Additional context

Among many issues the issue with regards to consistently evaluating the health of OpenSearch Dashboards, i.e. https://github.com/opensearch-project/helm-charts/issues/328

[sastorsl]

Further discussions in https://github.com/opensearch-project/helm-charts/pull/172

[Flyingliuhub]

Thanks @sastorsl for the suggestion here, OpenSearch Dashboards expose API (:5601/api/status) already. Is it possible to share your use case here? so we can understand more here. And is this PR "OpenSearchDashboards - Add healthcheck probes to charts" addressing your issue ? https://github.com/opensearch-project/helm-charts/pull/329/files