testkube chart 1.5.31 not installing (mongo and api broken)

[gioppoluca]

trying to install testkube using helm chart with no success, here are the findings

To Reproduce Here are the values used:

affinity: {}
autoscaling:
  enabled: false
  maxReplicas: 100
  minReplicas: 1
  targetCPUUtilizationPercentage: 80
fullnameOverride: testkube
image:
  pullPolicy: Never
  repository: ''
  tag: ''
imagePullSecrets: []
ingress:
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web, websecure
  className: ''
  enabled: 'true'
  hosts:
    - host: testkube.k8s-svil01.nivolapiemonte.it
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls: []
mongodb:
  architecture: standalone
  persistence:
    enabled: true
    storageClass: "csi-storage-nas"
    size: 2Gi
  auth:
    enabled: false
  enabled: true
  fullnameOverride: testkube-mongodb
  nameOverride: mongodb
  resources:
    requests:
      cpu: 150m
      memory: 100Mi
  service:
    clusterIP: ''
    nodePort: false
    type: ClusterIP
    port: '27017'
    portName: mongodb
nameOverride: testkube
nodeSelector: {}
podAnnotations: {}
podSecurityContext: {}
preUpgradeHook:
  name: mongodb-upgrade
  serviceAccount:
    create: true
replicaCount: 1
securityContext: {}
service:
  port: 80
  type: ClusterIP
serviceAccount:
  annotations: {}
  create: true
  name: ''
testkube-api:
  analyticsEnabled: true
  cliIngress:
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/configuration-snippet: |
        more_set_headers "X-CLI-Ingress: true";
      nginx.ingress.kubernetes.io/force-ssl-redirect: 'false'
      nginx.ingress.kubernetes.io/rewrite-target: /$1
      nginx.ingress.kubernetes.io/ssl-redirect: 'false'
    enabled: false
    hosts:
      - demo.testkube.io
    oauth:
      clientID: ''
      clientSecret: ''
      provider: github
      scopes: ''
    path: /results/(v\d/.*)
    tls:
      - hosts:
          - demo.testkube.io
        secretName: testkube-demo-cert-secret
    tlsenabled: 'false'
  fullnameOverride: testkube-api-server
  image:
    pullPolicy: Always
    repository: kubeshop/testkube-api-server
  minio:
    affinity: {}
    enabled: false
    image: minio/minio:latest
    nodeSelector: {}
    tolerations: []
  mongodb:
    dsn: mongodb://testkube-mongodb:27017
  nameOverride: api-server
  prometheus:
    enabled: false
  resources:
    requests:
      cpu: 200m
      memory: 100Mi
  service:
    port: 8088
    type: ClusterIP
  slackToken: ''
  storage:
    SSL: false
    accessKey: minio123
    accessKeyId: minio
    endpoint: ''
    endpoint_port: '9000'
    location: ''
    scrapperEnabled: true
    token: ''
  uiIngress:
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/auth-signin: ''
      nginx.ingress.kubernetes.io/auth-url: ''
      nginx.ingress.kubernetes.io/cors-allow-credentials: 'false'
      nginx.ingress.kubernetes.io/cors-allow-methods: GET
      nginx.ingress.kubernetes.io/enable-cors: 'true'
      nginx.ingress.kubernetes.io/force-ssl-redirect: 'false'
      nginx.ingress.kubernetes.io/rewrite-target: /$1
      nginx.ingress.kubernetes.io/ssl-redirect: 'false'
    enabled: false
    hosts:
      - demo.testkube.io
    path: /results/(v\d/executions.*)
    tls:
      - hosts:
          - demo.testkube.io
        secretName: testkube-demo-cert-secret
    tlsenabled: 'false'
testkube-dashboard:
  enabled: true
  fullnameOverride: testkube-dashboard
  image:
    pullPolicy: Always
    repository: kubeshop/testkube-dashboard
  ingress:
    annotations:
      traefik.ingress.kubernetes.io/router.entrypoints: web, websecure
      kubernetes.io/ingress.class: ''
    apiServerEndpoint: ''
    enabled: false
    hosts:
      - testkube-dash.k8s-svil01.nivolapiemonte.it
    path: /
    tls:
      - hosts:
          - testkube-dash.k8s-svil01.nivolapiemonte.it
        secretName: testkube-demo-cert-secret
    tlsenabled: 'false'
  nameOverride: dashboard
  oauth2:
    enabled: false
    env:
      clientId: ''
      clientSecret: ''
      cookieSecret: ''
      cookieSecure: 'false'
      githubOrg: ''
      redirectUrl: http://demo.testkube.io/oauth2/callback
    image:
      pullPolicy: Always
      repository: quay.io/oauth2-proxy/oauth2-proxy
      tag: latest
    name: oauth2-proxy
    path: /oauth2
    port: 4180
    selector: k8s-app
  service:
    port: 80
    type: ClusterIP
testkube-operator:
  imageProxy: gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0
tolerations: []

Expected behavior Working installation

Version / Cluster

• Which testkube version? 1.5.31
• What Kubernetes cluster?RKE1
• What Kubernetes version? 1.21

Screenshots

NAME                                                    READY   STATUS             RESTARTS   AGE
testkube-api-server-6d5dd9b4f-2zbcg                     0/1     CrashLoopBackOff   33         92m
testkube-dashboard-69ff578ffb-cbb87                     1/1     Running            0          92m
testkube-mongodb-6ddccbc68c-xh2g5                       0/1     Running            0          92m
testkube-operator-controller-manager-7fc67c5d68-j4glj   2/2     Running            0          92m

Additional context

minio wants a persistent volume but there is not the option to specify the storageclass so have to set minio to false and hope is not mandatory mongo is not installing, first seems ot is missing storageclass again but in this case there is the option to set it in persistance, but in the values of the chart is not present need to look at the mongochart and add values there ... those attributes sould be present in the testkube values.yaml file to help people know they have to look after also on that now I have mongo that try to do a network access on an amazon server do not know for what (I see that through network monitoring tools) and in any case is stuck in unready state logs state:

“{"t":{"$date":"2022-09-05T10:44:14.990+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"127.0.0.1:39740","uuid":"f7af44be-20a0-4cb8-a8e6-15a9ae1e650f","connectionId":96,"connectionCount":3}}
{"t":{"$date":"2022-09-05T10:44:14.993+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn96","msg":"client metadata","attr":{"remote":"127.0.0.1:39740","client":"conn96","doc":{"driver":{"name":"nodejs|mongosh","version":"4.4.0"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"5.4.0-72-generic"},"platform":"Node.js v14.19.1, LE (unified)","version":"4.4.0|1.3.1","application":{"name":"mongosh 1.3.1"}}}}
{"t":{"$date":"2022-09-05T10:44:19.756+00:00"},"s":"I", "c":"-", "id":20883, "ctx":"conn94","msg":"Interrupted operation as its client disconnected","attr":{"opId":3043}}"

API is crashloopbackoff with:

“{"level":"info","ts":1662375214.4073288,"caller":"slacknotifier/slack.go:184","msg":"initializing slack clientSLACK_TOKEN"}
I0905 10:53:35.477745 1 request.go:601] Waited for 1.006972911s due to client-side throttling, not priority and fairness, request: GET:https://10.43.0.1:443/apis/monitoring.coreos.com/v1alpha1?timeout=32s"

also I defined no slack token so I espect no usage of it

[ypoplavs]

Hello @gioppoluca, Thank you for reporting this!

Could you please provide us with the following information:

• what provider do you use to launch k8s? Is it AWS EKS, GCP etc?
• What network setup do you have?

Also do you have a default StorageClass in your k8s cluster?

[gioppoluca]

We use Rancher RKE1 in an on-premise environment We use cilium CNI We checked and there are no network deny rules that interfear. The K8S env is behind an enterprise proxy that is for sure blocking the calls to the outside but there should be no reason to the mongodb image to contact external network. This is a testing env and in prod cilium will deny egress rules from DB completely.

[ypoplavs]

Hello @gioppoluca,

We have updated our helm-charts with a Mongo version 11, since later versions introduce many bugs. Could you please install the latest chart version of testkube?

[gioppoluca]

chart is evolving fast, which is the stable one?

[ypoplavs]

Hello @gioppoluca , Please install 1.5.63 version of Testkube helm charts.

[vsukhin]

We moved to 1.7.30