opensearch-k8s-operator
opensearch-k8s-operator copied to clipboard
Published
20 hours ago •
opensearch-project
opensearch-project
Operator security: deploy with http certs only produce an error
hi all try to deploy next
apiVersion: opensearch.opster.io/v1
kind: OpenSearchCluster
metadata:
name: opensearch
spec:
confMgmt:
VerUpdate: false
autoScaler: false
monitoring: true
smartScaler: false
general:
serviceName: opensearch
version: 2.0.1
setVMMaxMapCount: true
serviceName: opensearch
httpPort: 9200
additionalConfig:
cluster.initial_master_nodes: opensearch-masters-0
compatibility.override_main_response_version: "true"
discovery.seed_hosts: opensearch-masters-0
security:
tls:
http:
generate: false
secret:
name: elasticsearch-server-cert
transport:
generate: true
dashboards:
enable: false
version: 2.0.1
replicas: 1
nodePools:
- component: masters
replicas: 1
resources:
limits:
cpu: 1100m
memory: 2Gi
requests:
cpu: 100m
memory: 1536Mi
jvm: "-Xmx1024M -Xms1024M"
roles:
- "data"
- "master"
diskSize: "16Gi"
persistence:
pvc:
storageClass: "ssd"
accessModes: [ReadWriteOnce]
and get error
[2022-07-18T14:07:15,031][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-masters-0] Exception while retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
org.opensearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
at org.opensearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:204) ~[opensearch-2.0.1.jar:2.0.1]
at org.opensearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(ClusterBlocks.java:190) ~[opensearch-2.0.1.jar:2.0.1]
at org.opensearch.action.get.TransportMultiGetAction.doExecute(TransportMultiGetAction.java:81) ~[opensearch-2.0.1.jar:2.0.1]
at org.opensearch.action.get.TransportMultiGetAction.doExecute(TransportMultiGetAction.java:58) ~[opensearch-2.0.1.jar:2.0.1]
at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:204) [opensearch-2.0.1.jar:2.0.1]
at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118) [opensearch-index-management-2.0.1.0.jar:2.0.1.0]
at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:202) [opensearch-2.0.1.jar:2.0.1]
at org.opensearch.security.filter.SecurityFilter.apply0(SecurityFilter.java:240) [opensearch-security-2.0.1.0.jar:2.0.1.0]
at org.opensearch.security.filter.SecurityFilter.apply(SecurityFilter.java:157) [opensearch-security-2.0.1.0.jar:2.0.1.0]
at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:202) [opensearch-2.0.1.jar:2.0.1]
at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78) [opensearch-performance-analyzer-2.0.1.0.jar:2.0.1.0]
at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:202) [opensearch-2.0.1.jar:2.0.1]
at org.opensearch.action.support.TransportAction.execute(TransportAction.java:174) [opensearch-2.0.1.jar:2.0.1]
at org.opensearch.action.support.TransportAction.execute(TransportAction.java:102) [opensearch-2.0.1.jar:2.0.1]
at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) [opensearch-2.0.1.jar:2.0.1]
at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) [opensearch-2.0.1.jar:2.0.1]
at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:423) [opensearch-2.0.1.jar:2.0.1]
at org.opensearch.client.support.AbstractClient.multiGet(AbstractClient.java:539) [opensearch-2.0.1.jar:2.0.1]
at org.opensearch.security.configuration.ConfigurationLoaderSecurity7.loadAsync(ConfigurationLoaderSecurity7.java:211) [opensearch-security-2.0.1.0.jar:2.0.1.0]
at org.opensearch.security.configuration.ConfigurationLoaderSecurity7.load(ConfigurationLoaderSecurity7.java:102) [opensearch-security-2.0.1.0.jar:2.0.1.0]
at org.opensearch.security.configuration.ConfigurationRepository.getConfigurationsFromIndex(ConfigurationRepository.java:375) [opensearch-security-2.0.1.0.jar:2.0.1.0]
at org.opensearch.security.configuration.ConfigurationRepository.reloadConfiguration0(ConfigurationRepository.java:321) [opensearch-security-2.0.1.0.jar:2.0.1.0]
at org.opensearch.security.configuration.ConfigurationRepository.reloadConfiguration(ConfigurationRepository.java:306) [opensearch-security-2.0.1.0.jar:2.0.1.0]
at org.opensearch.security.configuration.ConfigurationRepository$1.run(ConfigurationRepository.java:166) [opensearch-security-2.0.1.0.jar:2.0.1.0]
at java.lang.Thread.run(Thread.java:833) [?:?]
____________
[2022-07-18T14:07:23,050][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [opensearch-masters-0] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
how this can be resolved ?
I think @prudhvigodithi has been working on the 2.0.0 support - I wonder if this has been fixed in his work
Hey @elkh510 can you try with latest helm chart and update us?
helm install opensearch-operator opensearch-operator/opensearch-operator --version 2.0.0
also can you share the yaml file of elasticsearch-server-cert, so that I will try to replicate from my end.
Thank you
thank @dbason hi @prudhvigodithi
can you try with latest helm chart and update us?
same result
can you share the yaml file of elasticsearch-server-cert, so that I will try to replicate from my end.
maybe quick call ? if no, full example can be found below
---
apiVersion: v1
kind: Secret
metadata:
name: elasticsearch-server-ca-cert
type: Opaque
data:
ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURtVENDQW9HZ0F3SUJBZ0lVWWZPL1lvWGpCbm5Sb2JwZDVVN2lxeUFtc0Z3d0RRWUpLb1pJaHZjTkFRRUwKQlFBd1hERUxNQWtHQTFVRUJoTUNRMEV4RHpBTkJnTlZCQWdNQmxOQlRWQk1SVEVRTUE0R0ExVUVCd3dIUlZoQgpUVkJNUlRFTU1Bb0dBMVVFQ2d3RFQxSkhNUTB3Q3dZRFZRUUxEQVJWVGtsVU1RMHdDd1lEVlFRRERBUlNUMDlVCk1CNFhEVEl5TURjeU1EQTNNVEF5TUZvWERUSTRNREV4TURBM01UQXlNRm93WERFTE1Ba0dBMVVFQmhNQ1EwRXgKRHpBTkJnTlZCQWdNQmxOQlRWQk1SVEVRTUE0R0ExVUVCd3dIUlZoQlRWQk1SVEVNTUFvR0ExVUVDZ3dEVDFKSApNUTB3Q3dZRFZRUUxEQVJWVGtsVU1RMHdDd1lEVlFRRERBUlNUMDlVTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGCkFBT0NBUThBTUlJQkNnS0NBUUVBdFZWVXpxY0hEdWp5bzVsRTN3WDkwQ2E4aXpXZmRxQU0vcUppQnFveldqa0EKa3h5cCttT1dwaGJQWWdUR0FBR2VZeE5ZdEowOSthb0pmayt4STdXNld3eWZ2dzBWNjc2dzlITmJLZUpNcjRxUApoMGNoU3hWL1JzVktmNmNPR1NsYUcvMFBCNmYydHc2bnk2V1l4a0RtQnJ0TURtSWNzcHp6RE5RTDZTSmlIWStkCklXNVNvOFA5QnYzSEZRZWdwZnV3OGpWRFU3MGV2ZHJ1dHNlenZ3TE40SDh6d1kzVXFjSUFoVEdrc2UxM0krR1cKOURzY2Y3ZHZSbUZoRnRTQjlpUFdjSndmWEFEVDNVVlZBUFhyR1d4RUtqWHJFRUdRWXExR21yNXdzeDJBeVhwdgp4QUNxeTY0QW95K3V5anJ5UTZEMnVub2tCeUoza1VxYlZPd2FITzVwQlFJREFRQUJvMU13VVRBZEJnTlZIUTRFCkZnUVVKdkozWlhKbDEwZDB1Y1FzSUFqVGJrVzNtQUl3SHdZRFZSMGpCQmd3Rm9BVUp2SjNaWEpsMTBkMHVjUXMKSUFqVGJrVzNtQUl3RHdZRFZSMFRBUUgvQkFVd0F3RUIvekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBY3pPVgpJSUIzY0ZOSWpreGFhbXdGZXdLM3o4Mk5xQ2oyT3A5Z1N5cFdEL0xuTmlYRHB4Vk1heVBlbFJRSjRTblU0MElUCmZaVXJ6MWY3dzJBV2VMemx3L09YcFVCVVhldDNvUmZNNTZvTzNDWEdIR1RYU2s0S3VjYmljcHFTQjd0UGtyVEYKMTNlS3EwSjFaaDlHQTNURnlpNnpwZzVDNEhtUTZRanZRVG05YVpPWlU1UlhaMnhtOVBjNHkvV0dPeTNFeXlqYwpLWEEvci9ua2dPWmFsMjU5TllqQ3pzNTF5dE1MY290UG5jTFRsRXhjZ2dLcGZZZXFLdG5CaXMrTWJIS1k5WjdPCldRN2FDbnFqRUw3M1FLOUpITjBZS2ZiRlV5NDdJbzFkTkxYT3AzeUdJd0hFcHdZUVNPQS9HdHJkeGtSa1pNRnQKZk9sMEk4RnUvaVkzSElJSTN3PT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
ca.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBdFZWVXpxY0hEdWp5bzVsRTN3WDkwQ2E4aXpXZmRxQU0vcUppQnFveldqa0FreHlwCittT1dwaGJQWWdUR0FBR2VZeE5ZdEowOSthb0pmayt4STdXNld3eWZ2dzBWNjc2dzlITmJLZUpNcjRxUGgwY2gKU3hWL1JzVktmNmNPR1NsYUcvMFBCNmYydHc2bnk2V1l4a0RtQnJ0TURtSWNzcHp6RE5RTDZTSmlIWStkSVc1UwpvOFA5QnYzSEZRZWdwZnV3OGpWRFU3MGV2ZHJ1dHNlenZ3TE40SDh6d1kzVXFjSUFoVEdrc2UxM0krR1c5RHNjCmY3ZHZSbUZoRnRTQjlpUFdjSndmWEFEVDNVVlZBUFhyR1d4RUtqWHJFRUdRWXExR21yNXdzeDJBeVhwdnhBQ3EKeTY0QW95K3V5anJ5UTZEMnVub2tCeUoza1VxYlZPd2FITzVwQlFJREFRQUJBb0lCQUFWU0dxbnhwcmhlUjNoYgpiL1RuVlJvOWVMelE5dnZoR0dDRVhVSDZ6b1R3TlQ4YjBUVUNwUkg4aTF0QjdwbGpjc1ZQZFFQZnRValZIM0F3CllHOCt6dGdMRjhqZ1ZtdFRiaGZPSnJlazUwRWpYTERJZW9hY2U5NlV2V25mZWY5dEFqTGVDd25iZU5iSVBFNHkKYnJncWdWVmpPNGtVZ2VrUHgrWHhkdVFCV1VXcXUzaU44VkFzSDI1Tk96U1Y1dForY2kzS2g1RXhCR3NjM0JJZQptV0dvdklNQ21UMkxqYlZOaFdPNU4reVAwLzdza0VYMzB3UzUzWTRHb0hld3dob1dOM0FDdGQ0a3F6SkFKVDRDCnZIMXBVQmZrcXFXOXMvYkxQRTBycVhKcjJzZHFvTlVDanBDMlk5TXhxUlgyaEl0dnRiMkR3QXFOY282M3JjSkUKS283UnpNRUNnWUVBM1VJQmdZNHp6WThqZ0dZVzNlWWUzaDVGY3BDd2I5UG9tUUhqaWkvdjRNM1hjNEhxZGRzdQpSNzJRSG56ci93b1hDUTg5Z2xEZXNaOEtyZ2RzQ3k1TERPTXN5SzlsZUpDNzNoYUpLOWdmLzZuL1E1TWRjeHQvCnc4T1hEdXBWWlFaVWlENm52OTdVdHRBakhvY09XQndGcVRRVlN4bU1HOGhXbVl2WlJpOEh0SFVDZ1lFQTBjNTIKalhEMldHb05kanZGMW9oUHdSYzBNYmJiZXBRTE5HMlVmSmcrYjIreDkrQTIrUHcxWUh3emVmMGdSNjBXSktlYgpXNHB6eTJGS1l1QmlnRURRdXVEaWdoZXppazJPZFNvUHdrd2JlZjFpSFVBZmZvYkkvU0RvLzBLdkwvWVVzcmZpCkYrSzh1bDdtTDZuZDI0Wjl1Ym5aSC9vSk95VzF4aWNSNjF4N0VGRUNnWUIySndwUTJkUTJBQ3NRSlBRdkdQVUEKM2g5dDEzQXd0M090U1lIdHlEMkx5Y0EwOFhwMzUweW1pTWNPVU4xaGUwemp6SnluRzNKZ1hiTEJUanRYR1NpQQp3RXVvKzJqdllqTEFuVDR4Q0ovRjBrejdFL09WQldsTDdBV2R0R3RYb0hMQnN0dTNyVkhnTWZPZkdMWURQU3VVCnVMaHFPMEZEa004Vi9jVlFJa3MyQlFLQmdRREp0UVEzUVhZbFZIaHl4azZTeldSWDBGWW13WlZLeU0vbzdMMEgKK0ZwMlNsMUFZVGlZc2VralJPVThqOFljQXVVNkFrV1RnWTJINTVaYmJ6eGo3Mng5YmoxMk14RXlreE8xd3VWNQp1Qy9tWFZ1Wm9ZOWwxbkdmSXFlKzJUNlNEclNoaWJYOW5zZ3hLWkdHRzNxblZpckRoMUNldElWeGJ2VE9kN3Q3Cm11TFpzUUtCZ0NBQytueHF6YkpZcXpYQXdjdE1XMy9maW5mZnR3VmpBSUpWSUFSbGNMWGNoUFM1Vmgvai9URnIKOFRqc05qZGg0aXc1dm5oTld5RnpDUHJmTlNkWlRuS0o4T0VReUdYdFl6b2RFdjBPNjlOWFl3MXEzZUZVMGZ2WApqcDNKL3BkaFJoaC9lVklpOEVmRkQvZ3E4cjFZc0ZZb3FUOXkrUGV5R3NzNi9LaWIxUHBaCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
---
apiVersion: opensearch.opster.io/v1
kind: OpenSearchCluster
metadata:
name: opensearch
spec:
confMgmt:
VerUpdate: false
autoScaler: false
monitoring: true
smartScaler: false
general:
serviceName: opensearch
version: 2.1.0
setVMMaxMapCount: true
serviceName: opensearch
httpPort: 9200
additionalConfig:
cluster.initial_master_nodes: opensearch-masters-0
compatibility.override_main_response_version: "true"
discovery.seed_hosts: opensearch-masters-0
security:
tls:
http:
generate: true
caSecret:
name: elasticsearch-server-ca-cert
transport:
generate: true
caSecret:
name: elasticsearch-server-ca-cert
dashboards:
enable: false
version: 2.1.0
replicas: 1
nodePools:
- component: masters
replicas: 1
resources:
limits:
cpu: 1100m
memory: 2Gi
requests:
cpu: 100m
memory: 1536Mi
jvm: "-Xmx1024M -Xms1024M"
roles:
- "data"
- "master"
diskSize: "16Gi"
persistence:
pvc:
storageClass: "platform-elasticsearch-ssd"
accessModes: [ReadWriteOnce]
certificates generate commands
# Root CA
openssl genrsa -out root-ca-key.pem 2048
openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/C=CA/ST=SAMPLE/L=EXAMPLE/O=ORG/OU=UNIT/CN=ROOT" -out root-ca.pem -days 2000
# Node cert
openssl genrsa -out node1-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in node1-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out server.key
openssl req -new -key server.key -subj "/C=CA/ST=SAMPLE/L=EXAMPLE/O=ORG/OU=UNIT/CN=elasticsearch" -out node1.csr
openssl x509 -req -in node1.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -out server.crt -days 2000 -sha256 -extfile <(printf "subjectAltName=DNS:localhost,DNS:elasticsearch-master,DNS:elasticsearch-master-headless,DNS:elasticsearch,DNS:opensearch-master,DNS:opensearch-master-headless,DNS:opensearch\nkeyUsage=critical,digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth,clientAuth\nbasicConstraints=critical,CA:FALSE\nauthorityKeyIdentifier=keyid,issuer")
Out of interest why are you setting the initial masters manually? This is all part of the operator setting up a cluster. I'm not convinced it's causing the issue, but it might be worthwhile removing that.
why are you setting the initial masters manually?
we need opensearch in single-node https://github.com/Opster/opensearch-k8s-operator/issues/100
I'm not convinced it's causing the issue, but it might be worthwhile removing that.
same error
