The initializer can't complete when there is more than one container running within the same pod

[mcandeia]

Brief summary

Symptom

When running more than one container together within the same pod as the initializer, the operator can't retrieve container logs because the container was not specified.

Explained

I'm trying to use a custom k6 runner pod with dapr configured, in order to get this running, I have to add the dapr annotations to the runner, as the following:

dapr.io/app-id: "tester-app"
dapr.io/enabled: "true"

As the initializer pod annotations are the same as specified for runners, the initializer pod will also contain the daprd container, leading in two different problems:

1. As k8s jobs won't complete until all containers are done, the dapr will make this fail because I have no control to shutdown the pod, and, as the per documentation, the correct way for doing this is calling sidecar shutdown.

A simple solution would be having different annotations for initializer and runners because the runner itself can do http requests on teardown but the initializer doesn't.

2. Even though a patched k6 version is used for shutting down the dapr sidecar whenever the initialization is complete, the operator fails as it is unable to read the pod logs.

Suggested solutions

1. Do not reuse the same annotations as used for the test runner
2. Specify the k6 container when reading pod logs

k6-operator version or image

latest

K6 YAML

apiVersion: k6.io/v1alpha1
kind: K6
metadata:
  name: k6-sample
spec:
  parallelism: 1
  script:
    configMap:
      name: tests
      file: test.js
  runner:
    metadata:
      annotations:
        dapr.io/app-id: "tester-app"
        dapr.io/enabled: "true"
    image: localhost:5001/k6-custom:latest

Other environment details (if applicable)

dapr installed

Steps to reproduce the problem

git clone github.com/mcandeia/k6-operator-dapr-error
cd k6-operator-dapr-error
./setup.sh

Check the manager logs:

2022-11-05T00:47:43.891Z        ERROR   controllers.K6  unable to stream logs from the pod      {"k6": "default/k6-sample", "error": "a container name must be specified for pod k6-sample-initializer-cqs8s, choose one of: [k6 daprd]"}

Expected behaviour

initializer should complete regardless of running containers

Actual behaviour

The test initializer can't complete

https://github.com/grafana/k6-operator/issues/164#tasklist-block-8de0afeb-65ae-40a0-87b0-bcd26f483ef3

[javaducky]

Suggested solutions

1. Do not reuse the same annotations as used for the test runner
2. Specify the k6 container when reading pod logs

Just adding context that item 2 is now resolved by the merge of #165. Handling of annotations is the remaining issue.

[yorugac]

Hi @mcandeia, there was an issue with Istio degradation due to annotations removal in #172 so I reverted the #166. My apologies for this. I'm re-opening your issue but I'd like to clarify this part with you:

Even though a patched k6 version is used for shutting down the dapr sidecar whenever the initialization is complete, the operator fails as it is unable to read the pod logs.

What do you mean by "patched k6 version" here? It sounds to me like with your fix for containers in #165 merged, it should actually be sufficient for your use case without changing annotations. Is that correct? I haven't yet the time to dig into dapr logic enough to evaluate this so I'd appreciate if you could clarify :slightly_smiling_face:

[mcandeia]

Hi @mcandeia, there was an issue with Istio degradation due to annotations removal in #172 so I reverted the #166. My apologies for this. I'm re-opening your issue but I'd like to clarify this part with you:

Even though a patched k6 version is used for shutting down the dapr sidecar whenever the initialization is complete, the operator fails as it is unable to read the pod logs.

What do you mean by "patched k6 version" here? It sounds to me like with your fix for containers in #165 merged, it should actually be sufficient for your use case without changing annotations. Is that correct? I haven't yet the time to dig into dapr logic enough to evaluate this so I'd appreciate if you could clarify 🙂

Hello @yorugac, by “patched k6” I meant a custom k6 runner image that shutdowns the sidecar when the initializer complete. But the fix was actually applied by having separate annotations for the initializer, did you open the repository link that I provided in the issue? I think you can reproduce there(you have to pin the k6 operator version now because there is pointing to latest)

[mcandeia]

In addition, the latest operator version is not working with dapr and I had to pin the 0.0.8 release.