File Based Settings for Elasticsearch

[grcevski]

Description

Introduction

Watches a directory under Elasticsearch config for changes to a file called settings.json. This is a JSON file describing the settings that the operator wants to set that will be immutable through the REST API. Any changes to these settings will only be allowed through modifying the file. As configured by default we watch for a path operator/settings.json under the config directory. The name of the directory operator can be changed via settings.

The File Settings Service will notice updates to the file and perform an update. The operator directory doesn't have to exist, the file setting service will notice when it's created. Recommended way to change operator file based settings is through modifying the settings.json file in a brand new location and then performing a symlink under the Elasticsearch config directory.

Initial support for File Based Settings was delivered in 8.4 through this PR: https://github.com/elastic/elasticsearch/pull/88329

Remaining work

• [x] Restore file settings state after snapshot restore (#89321)
• [x] /_cluster/settings (v8.4.0)
• [x] /_ilm/policy/<policy_id> (v8.4.0)
• [x] ​​/_security/role_mapping (https://github.com/elastic/elasticsearch/pull/89667)
• [x] /_autoscaling/policy (https://github.com/elastic/elasticsearch/pull/89708)
• [x] /_snapshot/cloud-snapshots (https://github.com/elastic/elasticsearch/pull/89601)
• [x] /_slm/policy (https://github.com/elastic/elasticsearch/pull/89567)
• [ ] /_ingest/pipeline/ (https://github.com/elastic/elasticsearch/pull/89735)
• [ ] /_index_template/ (https://github.com/elastic/elasticsearch/pull/90143)
• [ ] /_component_template/ (https://github.com/elastic/elasticsearch/pull/90143)

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[grcevski]

Complete.