fix(env2yaml): Add monitoring.cluster_uuid

[djboris9]

Release notes

Fixed environment based configuration for monitoring.cluster_uuid

What does this PR do?

This commit adds monitoring.cluster_uuid to the env2yaml configuration whitelist and enables the user to set this configuration option via environment variable. It fixes #12211.

Why is it important/What is the impact to the user?

This PR makes the environment based configuration more usable.

Checklist

• [X] My code follows the style guidelines of this project
• [X] I have commented my code, particularly in hard-to-understand areas
• [X] I have made corresponding changes to the documentation
• [X] I have made corresponding change to the default configuration files (and/or docker env variables)
• [ ] ~~I have added tests that prove my fix is effective or that my feature works~~

Author's Checklist

N/A

How to test this PR locally

Build env2yaml and validate it by providing environment variables.

Related issues

Closes #12211

Use cases

N/A

Screenshots

N/A

Logs

N/A

[logstashmachine]

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

[cla-checker-service[bot]]

💚 CLA has been signed

[andsel]

Hi @djboris9 thanks for your contribution! Please sign the CLA using an email is the same used in the commits and that's associated with your Github account.

[djboris9]

Hi @andsel This was already done. Can you recheck it?

[andsel]

Hi @djboris9

now CLA seem ok, maybe there was some delay when I checked before

[andsel]

@djboris9 your contribution was accepted and merged.

Thanks for improving Logstash!

[rpasche]

@andsel , @djboris9 Sorry....but could you please tell, what is the exact environment variable name that needs to be set within logstash to get this working? I am currently using the official 8.4.0 docker image and in my tests, neither passing MONITORING_CLUSTER_UUID nor XPACK_MONITORING_CLUSTER_UUID does work. curl localhost:9600/_node does not provide the object .monitoring.cluster_uuid. The only way to get this working, is to truly set this within logstash.yml.

Thank you for your help.

[rpasche]

Just some output trying to use env2yaml

logstash@c4f9de3d0918:~$ env | grep -i cluster
XPACK_MONITORING_CLUSTER_UUID=NBSVsVarQIqUJ-hGivIHlw
MONITORING_CLUSTER_UUID="NBSVsVarQIqUJ-hGivIHlw"
logstash@c4f9de3d0918:~$ cat config/logstash.yml
http.host: 0.0.0.0
node.name: logstash
pipeline.ecs_compatibility: disabled
xpack.monitoring.elasticsearch.hosts:
- http://elasticsearch:9200
xpack.monitoring.enabled: false
logstash@c4f9de3d0918:~$ env2yaml config/logstash.yml
2022/09/02 06:57:22 Setting 'node.name' from environment.
2022/09/02 06:57:22 Setting 'xpack.monitoring.enabled' from environment.
2022/09/02 06:57:22 Setting 'pipeline.ecs_compatibility' from environment.

Partial output of :9600/_node

{
  "host": "c4f9de3d0918",
  "version": "8.4.0",
  "http_address": "0.0.0.0:9600",
  "id": "99d308fb-a5c2-45d0-9204-9124a7b1f5c8",
  "name": "logstash",
  "ephemeral_id": "28831551-484a-4b80-a57a-f8568a179994",
  "status": "green",
  "snapshot": false,
  "pipeline": {
    "workers": 2,
    "batch_size": 125,
    "batch_delay": 50
  },
  "pipelines": {
    "p1": {
...

[djboris9]

Hi @rpasche

You need to use MONITORING_CLUSTER_UUID as in this example (built from main):

$ touch test.yaml
$ MONITORING_CLUSTER_UUID=foobar ./env2yaml test.yaml
2022/09/02 11:43:19 Setting 'monitoring.cluster_uuid' from environment.
$ cat test.yaml 
monitoring.cluster_uuid: foobar

This PR didn't made it into 8.4.0 but should be included into 8.5.0

[rpasche]

@djboris9 Thanks. I just confirmed it myself. I compiled env2yaml locally and it works as expected. But I also saw, that the code also did not make it into the 8.4.1 version of the logstash docker image.

[andsel]

Hi @rpasche this improvements is planned to be released with version 8.5.0 and not backported to 8.4.x

[rpasche]

Hi @andsel , sad to hear. Especially, as this seems - to me - to be a very simple "external" tool that should be very easy to backport.

But thanks.

[andsel]

@logstashmachine backport 8.4

[andsel]

@logstashmachine backport 7.17

[andsel]

@logstashmachine backport 7.17

[andsel]

@rpasche thanks for noting that, this is going to be released also in 8.4.2 and 7.17.7

[rpasche]

@andsel thank you for backporting. Really appreciate it.