[BUG] Internal User changes to OpenSearch not reflected in config files

[ks4225]

Describe the bug The passwords that are set for internal users in OpenSearch are not reflected in the config files (i.e. internal_users.yml). Therefore, using securityadmin.sh will end up reverting the users back to their default config and passwords will no longer match what's in the cluster config YAML file.

How to reproduce Steps to reproduce the behavior:

1. Build an epiphany cluster with at least 1 es node with passwords set for the internal users (in cluster config YAML)
2. Run the ./securityadmin.sh to apply some kind of security change
3. Attempt to use the credentials from the cluster config YAML to authenticate in ES

Expected behavior We would expect that the password changes will persist through running ./securityadmin.sh.

Config files If applicable, add config files to help explain your problem.

Environment

• Cloud provider: Azure (but expect it to affect ALL)
• OS: Ubuntu 18.04.4 LTS

epicli version: [1.0.0]

Additional context Not sure if this is strictly a bug, but it is unexpected behaviour. We're currently using our own logic to update internal_users.yml to match the config YAML.

---

DoD checklist

• Changelog
  • [ ] updated
  • [ ] not needed
• COMPONENTS.md
  • [ ] updated
  • [ ] not needed
• Schema
  • [ ] updated
  • [ ] not needed
• Backport tasks
  • [ ] created
  • [ ] not needed
• Documentation
  • [ ] added
  • [ ] updated
  • [ ] not needed
• [ ] Feature has automated tests
• [ ] Automated tests passed (QA pipelines)
  • [ ] apply
  • [ ] upgrade
  • [ ] backup/restore
• [ ] Idempotency tested
• [ ] All conversations in PR resolved

[to-bar]

@ks4225 I was not aware that you run securityadmin.sh. This script is run automatically to initialize the .opendistro_security index and configure authentication and authorization methods by postInstall package's task (via install_demo_configuration.sh).

But this is demo configuration so one may need to customize it.

Could you provide some details about settings you customize via YAML files?

According to docs:

The best use of these YAML files is to configure reserved and hidden resources, such as the admin and kibanaserver users. You might find it easier to create other users, roles, mappings, action groups, and tenants using Kibana or the REST API.

I see the following options, your comment is welcome:

1. Epiphany will propagate specified passwords to internal_users.yml.
2. You will perform customization via REST API or Kibana (if possible) without running securityadmin.sh.
3. Epiphany will extend configurable settings to handle and apply your customization to the YAML files in plugins/opendistro_security/securityconfig or/and via REST API.

[ks4225]

Essentially, we need to enable anonymous access for ES, which the app currently relies on. There are some other minor tweaks. I'll summarise here, but could also share our playbook if that's easier.

• In elasticsearch.yml - 'opendistro_security.ssl.http.enabled: false' (although this is probably not part of securityadmin.sh)
• In config.yml - ' anonymous_auth_enabled: true'
• In roles.yml -

block: |
              opendistro_security_anonymous:
                reserved: true
                cluster_permissions:
                  - cluster:monitor/*
                  - CLUSTER_COMPOSITE_OPS_RO
                  - "indices:data/read/scroll"
                  - "indices:data/read/scroll/*"
                  - "indices:data/write/bulk"
                index_permissions:
                  - index_patterns:
                      - "*"
                    allowed_actions:
                      - "read"
                      - "indices:admin/*"
                      - "indices:monitor/*"
                      - "indices:admin/mappings/*"
                      - "indices:admin/mapping/*"
                      - "indices:data/write/*"

• In roles.yml -

block: |
              ##DE Kibana Full Access Admin Role
              DE_Kibana_Full_Role:
                reserved: true
                hidden: false
                description: "Allow full access to all indices and all cluster APIs"
                cluster_permissions:
                  - "*"
                index_permissions:
                  - index_patterns:
                      - "*"
                    allowed_actions:
                      - "*"
                tenant_permissions:
                  - tenant_patterns:
                      - "*"
                    allowed_actions:
                            - "kibana_all_write"

• in roles_mapping.yml -

block: |
              opendistro_security_anonymous:
                reserved: false
                backend_roles:
                  - opendistro_security_anonymous_backendrole
                users:
                  - "opendistro_security_anonymous"

• in roles_mapping.yml -

block: |
              DE_Kibana_Full_Role:
                reserved: false
                backend_roles:
                  - "DE_Kibana_Fullaccess"
                  - "opendistro_security_anonymous_backendrole"
                description: "Maps DE_Kibana_Full_Role to DE_Kibana_Fullaccess from Keycloak"

We have logic in our playbook, which I imagine as a solution you could use as option 1 you mention. We simply generate a hash (/usr/share/elasticsearch/plugins/opendistro_security/tools/hash.sh) corresponding the the passwords in the cluster config and write them to the internal_users.

Option 2 also makes sense, we just haven't had the scope/time to explore that conversion process. It has worked up till our upgrade to Epiphany 1.0.0, so it hasn't been required.

Option 3 is interesting and will reduce the extra config we apply.

[seriva]

We should do this in context of OpenSearch first, backport to Opendistro can be decided once that is completed.