elastic
Permissions/Users for Beats products - Revamp documentation and/or product behavior
The following statement refers to 8.14.1 (likely affects previous versions).
I'm mostly focusing on the setup and publisher users/roles.
- The different Beats reuse the same documentation (e.g. https://www.elastic.co/guide/en/beats/packetbeat/current/privileges-to-setup-beats.html) for
setup.
In the setup role, we suggest the user to define a custom role, plus reuse a set of built-in roles (ingest_admin and kibana_admin). The ingest_admin hides a wide permission set manage_ilm, manage_pipelines, manage_templates).
It's worth mentioning that (at least from my analysis) only Filebeat, Packetbeat and Winlogbeat actually use the manage_pipelines permission. The other Beats do not rely on any out-of-the-box ingest pipeline.
Proposal for the setup user docs
- we might replace the use of the
ingest_adminby explicitclusterpermissions. It would also allow to reduce the permissions tomanage_ilmandmanage_templatesfor all the Beats (except Winlogbeat, Filebeat, Packetbeat).
- The different Beats are "correctly" declaring the permissions needed (e.g. https://www.elastic.co/guide/en/beats/packetbeat/current/privileges-to-publish-events.html) for
publishing.
In the publishing role:
2a) For all the Beats, we declare only read_ilm permission (and we tell that we can even remove it if we specify in the config setup.ilm.check_exists: false), which is correct
2b) For Winlogbeat, Filebeat, we declare also the read_pipelines permission as those Beats are ALWAYS checking for the presence of the ingest pipelines and attempt to install them if they're not there, at each connection to Elasticsearch.
There is no setting to disable this behavior. Packetbeat NEEDS the permission since 8.13.0 onwards.
Not granting the read_pipelines role, those Beats will still attempt to install the pipelines (cannot read the pipeline, then it's not present, so Beats attempts to reinstall).
Note: there's actually code to handle overwrite_pipelines as configuration, but it seems never read from the configuration (https://github.com/elastic/beats/blame/main/filebeat/config/config.go#L50C51-L50C70).
I imagine this has been done to better support Beat config reload, so that if someone enables a module, the ingest pipelines are loaded on config reload while re-initializing the connections.
2c) For all Beats, they attempt to check the Index Template. It seems the monitor permission is allowing reading the index templates installed. To completely disable the check, one could suggest to use setup.template.enabled: false. The monitor permission would be still needed for the license check, so not a lot to gain here.
Possible proposal for publishing user
- In the docs, add the
read_pipelinespermission for Packetbeat on 8.13.0 onwards. - In the docs, explain why we need
read_pipelinesmore clearly for Winlogbeat, Packetbeat, Filebeat - [Requires code change] Introduce a new setting
setup.pipelines.*(defaulting totrue), which might then open to removing theread_pipelinepermission in case the user sets it tosetup.pipelines.enabledtofalse. Whensetup.pipelines.enabledis set tofalse, I would log an INFO telling which pipelines would be necessary.
- The API Key docs for Beats have some issues:
- The
setupAPI Key is missing completely - The
publishingAPI Key is defaulting grantingread_pipelinealso to Beats which do not need such role (e.g. https://www.elastic.co/guide/en/beats/metricbeat/current/beats-api-keys.html)
Suggestions for the API Key
- Provide the
setup,publisherandmonitoringAPI Key version for the Beats - Reduce the permissions when possible (
read_pipelineshould be used only on Winlogbeat, Filebeat, Packetbeat)
Obviously the code change is likely not going to happen as Elastic Agent solved this kind of problems with the introduction of integration assets, but worth reporting it here.
