TLS handshake error in opentelemetry-operator

[sunilkumar-nfer]

Hi All,

I am very new to open telemetry, i was deploying the operator using this link.

but after deploying from the operator pod am seeing these logs in the pod http: TLS handshake error from x.x.x.x:50516,

but i can see from API server request is received and connection establish from the operator pod.

kube-api server logs

I0626 05:36:00.137545       1 client.go:354] "Received DIAL_REQ" serverID="ed60f6a8-2b5e-4f2b-bf8b-cad4738db" agentID="88bc09a7-e124-447c-b8aa-fabe5825" dialID=8045767835342242194 dialAddress="pod-ip:9443"
I0626 05:36:00.138933       1 client.go:429] "Endpoint connection established" dialID=804576783242194 connectionID=320 dialAddress="pod-ip:9443"

i already try with both cert manager, auto generate certificate and parsing own certificates. but in every case we are receiving same issue.

using this values file for operator

opentelemetry-operator:
  fullnameOverride: optel-operator
  manager:
    image:
      repository: our-registery-url/nference/opentelemetry/opentelemetry-operator
      tag: "1.0.0"
    collectorImage:
      repository: "our-registery-url/opentelemetry/opentelemetry-collector-k8s"
      tag: 0.102.1
    autoInstrumentationImage:
      python:
        repository: "our-registery-url/opentelemetry/autoinstrumentation-python"
        tag: "1.0.0"
    
    resources:
      limits:
        cpu: 100m
        memory: 128Mi
        # ephemeral-storage: 50Mi
      requests:
        cpu: 100m
        memory: 64Mi
  
  kubeRBACProxy:
    enabled: true
    image:
      repository: our-repo-url/opentelemetry/kube-rbac-proxy
      tag: v0.15.0
    ports:
      proxyPort: 8443
    resources:
      limits:
        cpu: 500m
        memory: 128Mi
      requests:
        cpu: 50m
        memory: 64Mi
  
  admissionWebhooks:
    create: true
    servicePort: 443
    failurePolicy: Fail
    secretName: ""

    certManager:
      enabled: create
    
    autoGenerateCert:
      enabled: false
      recreate: false

helm version:- 3.14 Kubernetes version:- 1.28 Go version:- go1.21.9 kubectl:- 0.26.11 chart-version- 0.62.0

i am not sure what i am doing wrong here, can some help here as we need to work on tracing with Auto-instrumentation