cli: add hidden command-line flag for overriding settings

[yaauie]

Release notes

[rn: skip]

What does this PR do?

When invoking Logstash in test, it is often necessary to override a setting, but cumbersome to provide a whole logstash.yml (or to append specific overrides to a fixture-provided one) when there is no provided command-line option paired with the setting.

This feature adds a hidden --setting option with an -S shortname, whose value is a colon- or equals-separated key-value pair.

• long-form single arg: --setting=queue.type:persisted
• long-form arg-pair: --setting node.name:myname
• short-form: -Squeue.type=persisted

Why is it important/What is the impact to the user?

No user impact; improved testability.

Checklist

• [x] My code follows the style guidelines of this project
• [x] I have commented my code, particularly in hard-to-understand areas
• ~~[ ] I have made corresponding changes to the documentation~~
• ~~[ ] I have made corresponding change to the default configuration files (and/or docker env variables)~~
• [x] I have added tests that prove my fix is effective or that my feature works

How to test this PR locally

• Invoke logstash with any setting in any of the supported formats and observe that the modified setting value is in the debug output:

  • short-form -Skey=value:

    bin/logstash --log.level=debug -Snode.name=banana | grep 'node.name'
    

    [2025-04-23T16:39:57,851][DEBUG][logstash.runner          ] *node.name: banana (default: perhaps)
    

  • long-form arg pair --setting key:value:

    bin/logstash --log.level=debug --setting node.name:orange | grep 'node.name'
    

    [2025-04-23T16:40:01,001][DEBUG][logstash.runner          ] *node.name: orange (default: perhaps)
    

  • long-form single arg --setting=key:value:

    bin/logstash --log.level=debug --setting=node.name:grape | grep 'node.name'
    

    [2025-04-23T16:40:04,571][DEBUG][logstash.runner          ] *node.name: grape (default: perhaps)
    

• Invoke with a non-coercible value to observe a helpful error message:

  bin/logstash --log.level=debug -Sapi.enabled=banana
  

  ERROR: failed to apply setting `api.enabled=banana`: Cannot coerce `banana` to boolean (api.enabled)
  

• Invoke with an unregistered setting name to observe a helpful error message:

  bin/logstash --log.level=debug -Sfruit.favorite=banana
  

  ERROR: failed to apply setting `fruit.favorite=banana`: unknown setting `fruit.favorite`
  

Use cases

Simplifies testing by allowing integration test invocation to include any setting without needing to override the settings.yml that may or may not be provided by a fixture.

Logs

      arbitrary settings
        --setting=key:value formatted options
          when specifying `--setting=node.name:my-fancy-node-name --setting=api.enabled:false`
            overrides setting `node.name` to `my-fancy-node-name`
            overrides setting `api.enabled` to `false`
          when specifying `--setting=api.enabled:false`
            overrides setting `api.enabled` to `false`
          when specifying uncoercible `--setting=api.enabled:yellow`
            raises a helpful usage error about failed coercion
          when specifying unregistered `--setting=banana:yellow`
            raises a helpful usage error about the unknown setting
          when specifying `--setting=node.name:my-fancy-node-name`
            overrides setting `node.name` to `my-fancy-node-name`
        -Skey=value formatted options
          when specifying unregistered `-Sbanana=yellow`
            raises a helpful usage error about the unknown setting
          when specifying `-Snode.name=my-fancy-node-name -Sapi.enabled=false`
            overrides setting `node.name` to `my-fancy-node-name`
            overrides setting `api.enabled` to `false`
          when specifying uncoercible `-Sapi.enabled=yellow`
            raises a helpful usage error about failed coercion
          when specifying `-Snode.name=my-fancy-node-name`
            overrides setting `node.name` to `my-fancy-node-name`
          when specifying `-Sapi.enabled=false`
            overrides setting `api.enabled` to `false`
        --setting key:value formatted options
          when specifying `--setting api.enabled:false`
            overrides setting `api.enabled` to `false`
          when specifying `--setting node.name:my-fancy-node-name --setting api.enabled:false`
            overrides setting `api.enabled` to `false`
            overrides setting `node.name` to `my-fancy-node-name`
          when specifying unregistered `--setting banana:yellow`
            raises a helpful usage error about the unknown setting
          when specifying `--setting node.name:my-fancy-node-name`
            overrides setting `node.name` to `my-fancy-node-name`
          when specifying uncoercible `--setting api.enabled:yellow`
            raises a helpful usage error about failed coercion

[mergify[bot]]

This pull request does not have a backport label. Could you fix it @yaauie? 🙏 To fixup this pull request, you need to add the backport labels for the needed branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
• If no backport is necessary, please add the backport-skip label

[elastic-sonarqube[bot]]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[donoghuc]

Looks like this handles bool, numeric, string. Are more complex setting types handled? For example can api.ssl.supported_protocols (appears to support accepting a list) be set on CLI?

[yaauie]

Looks like this handles bool, numeric, string. Are more complex setting types handled? For example can api.ssl.supported_protocols (appears to support accepting a list) be set on CLI?

:thinking: it looks like we have a SplittableStringArray setting implementation that could handle comma-delimited input in theory, but it doesn't support an allow-list like the StringArray implementation used by api.ssl.supported_protocols.

We also have other settings based on ArrayCoercible in monitoring and config management.

I'd rather not make this a blocker for a hidden primarily-for-test feature.

[elastic-sonarqube[bot]]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[elasticmachine]

:green_heart: Build Succeeded

• Buildkite Build
• Commit: d560441d166bfaf27f2f7b26d509a80cfed58cef

History

• :yellow_heart: Build #2866 was flaky 3002abdeb1c9e40c271bcb0da6510d8f723bed3e