x509: certificate signed by unknown authority

[kaykhan]

I am using ECK with fleet and agents. I have setup the Kibana Agent and set the host, username and password and left the certificate entry empty.

However i get the following certificate error in the agent logs

{"log.level":"error","@timestamp":"2024-10-17T08:30:37.777Z","message":"Error fetching data for metricset kibana.cluster_actions: error making http request: Get "https://kibana-prod-eck-kibana-kb-http.elastic-system.svc:5601/api/status": x509: certificate signed by unknown authority","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"kibana/metrics-default","type":"kibana/metrics"},"log":{"source":"kibana/metrics-default"},"log.origin":{"file.line":256,"file.name":"module/wrapper.go","function":"github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).fetch"},"service.name":"metricbeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}I

As far as i understand ECK should use self signed certificates.

kibana config

    version: 8.15.2
    spec:
      count: 1
      elasticsearchRef:
        name: elasticsearch-prod-eck-elasticsearch
      http:
        service:
          spec:
            type: NodePort
      podTemplate:
        metadata:
          labels:
            scrape: kb
        spec:
          containers:
          - name: kibana
            resources:
              limits:
                memory: 2Gi
                cpu: 1
          tolerations:
            - key: "karpenter/elastic"
              operator: "Exists"
              effect: "NoSchedule"
          nodeSelector:
            karpenter-node-pool: elastic
            karpenter.k8s.aws/instance-size: large
      config:
        xpack.encryptedSavedObjects:
            encryptionKey: wmtk5CT8qsIn31WSXmd0zPvUrDSvezpJF5gHq4c+cDNbOVJDXHmMBl+537PdUHLx
        xpack.fleet.agents.elasticsearch.hosts: ["https://elasticsearch-prod-eck-elasticsearch-es-http.elastic-system.svc:9200"]
        xpack.fleet.agents.fleet_server.hosts: ["https://fleet-server-prod-eck-fleet-server-agent-http.elastic-system.svc:8220"]
        xpack.fleet.packages:
          - name: elastic_agent
            version: latest
          - name: fleet_server
            version: latest
          - name: kibana
            version: latest
          - name: kubernetes
            version: latest
          - name: system
            version: latest
          - name: apm
            version: latest
          - name: elasticsearch
            version: latest
        xpack.fleet.agentPolicies:
          - name: Fleet Server on ECK policy
            id: eck-fleet-server
            namespace: default
            monitoring_enabled:
              - logs
              - metrics
            unenroll_timeout: 900
            package_policies:
            - name: fleet_server-1
              id: fleet_server-1
              package:
                name: fleet_server
          - name: Elastic Agent on ECK policy
            id: eck-agent
            namespace: default
            monitoring_enabled:
              - logs
              - metrics
            unenroll_timeout: 900
            is_default: true
            package_policies: 
              - id: kibana-1
                name: kibana-1
                package:
                  name: kibana
              - name: kubernetes-1
                id: kubernetes-1
                package:
                  name: kubernetes
              - id: system-1
                name: system-1
                package:
                  name: system
              - id: apm-1
                name: apm-1
                package:
                  name: apm
              - id: elasticsearch-1
                name: elasticsearch-1
                package:
                  name: elasticsearch