k6-operator icon indicating copy to clipboard operation
k6-operator copied to clipboard

Feature request: can be mount ConfigMap/Secret to k6 pods

Open mugioka opened this issue 3 years ago • 10 comments

motivation

I am dealing with binaries (jpeg images) in k6 script as shown below. However, I am having trouble mounting the volume to k6 pods with my current K6 CR.

import http from 'k6/http';
import { check, group, sleep } from 'k6';

export const options = {
  vus: 10,
  duration: '300s',
  iterations: 100000,
};

const SLEEP_DURATION = 0.1;

const miniImgFile = open('./test-mini.jpg', 'b');

const bigImgFile = open('./test-big.jpg', 'b');

export default function () {
  // omit
}

mugioka avatar Jun 30 '22 10:06 mugioka

Hi @mugioka, Sorry for the delay. Thank you for both the issue and the PR! Could you please explain why volumeClaim option doesn't work for you? (I've just added a section to the README for it, since it wasn't very explicit before: here)

yorugac avatar Jul 08 '22 15:07 yorugac

Thank you for reply @yorugac If I understand correctly, it is my understanding that the VC specified in the README can only be used to mount k6 scenario. If I want to mount images base64 encoded and mounted in a container, I need to change this PR.

mugioka avatar Jul 15 '22 01:07 mugioka

If I understand correctly, it is my understanding that the VC specified in the README can only be used to mount k6 scenario.

No, it doesn't have such limitations, it's just a normal volume. More precisely, runner does expect to find the script there but you can store any other data as well, e.g. any additional files like JSONs or images, etc. If the script uses those files, then k6 would reference the mounted folder the same way it does normally. Example:

# volume contains:
test.js
users.json

Then your test.js can load that JSON as JSON.parse(open("./users.json"))

yorugac avatar Jul 15 '22 13:07 yorugac

Oh, sorry, my understanding was incorrect. But I think it would be better to be able to use Secret or something like that to securely mount the GCP Service Account key json and other confidential information to k6 runner, what do you think?

mugioka avatar Jul 15 '22 13:07 mugioka

@mugioka I see, I was looking at your PR and was thinking about volumes themselves mostly. But if the problem is with secrets, then secretKeyRef should work even now: https://github.com/grafana/k6-operator/blob/c63b127b53ca18ddf98f1648e25aefb785efe207/config/crd/bases/k6.io_k6s.yaml#L791 So it should be possible to access Secret as an env var. It looks like it is present in envFrom as well, by the way.

Also, if the key is in ServiceAccount, it can be passed directly as described here: https://github.com/grafana/k6-operator/blob/main/README.md#serviceaccount

Those are the options now, I believe.

yorugac avatar Jul 15 '22 14:07 yorugac

@yorugac I apologize if my understanding is wrong, but I was referring to the GCP service account as an example (https://stackoverflow.com/a/47023291), not the kubernetes service account. However, it does not seem to be a good example, since the credentials for the GCP service account can be replaced by obtaining a temporary token using Workload Identity. Another example is the case where I want to mount a client certificate (used for communication when performing load testing) on a container, I still think it would be better to be able to use Secret. This may be an unusual case, but in my environment, I would like to mount Secret (as a file) in a container in the future for a use similar to the above. What do you think?

mugioka avatar Jul 15 '22 15:07 mugioka

Hi @mugioka, sorry for the delay. So regarding your example, it really depends on the exact requirements of your setup. E.g. you could put it into an image and use security context for test's execution - maybe that's sufficient? I don't think there is a silver bullet here, it's a matter of the product requirements.

FYI, we have something similar planned for Vault support: #103 At the moment, I think this kind of support would allow to load all kinds of secret data. But this is still a very early stage research, and I cannot say yet how addition of this support would correlate with simple passing of volumes as you suggest.

I'm also rather curious if there's additional interest in this kind of features.

yorugac avatar Jul 22 '22 15:07 yorugac

@yorugac

maybe that's sufficient?

Yes, it is feasible to do so. However, I would prefer to be able to VolumeMount the Secret, as I find it less convenient to have to build a custom docker image when I just want to include sensitive values in a container.

FYI, we have something similar planned for Vault support: https://github.com/grafana/k6-operator/issues/103

This is a good feature for those who manage sensitive values outside the repository (Vault). However, in my environment, I use sops to encrypt sensitive values and manage in the repository, so I would prefer to be able to VolumeMount Secret in a container.

mugioka avatar Jul 26 '22 08:07 mugioka

@yorugac I'm running into this need as well.

I scenario is that our CI/CD is making our custom K6 modules and loading it onto the PVC so that the Runners have the latest readily available. As of now I see no way to

import { check, group, sleep } from 'k6';
import http from 'k6/http';
import { uuidv4 } from 'https://jslib.k6.io/k6-utils/1.4.0/index.js';
import { generateEnrollment } from 'v1/generators/enrollment.js'; # i cannot get any path to work

Ive also found a related ticket here https://github.com/grafana/k6-operator/issues/121

kneemaa avatar Aug 09 '22 20:08 kneemaa

I'm also running into this issue

If we want to use mTLS client certs in the k6 job container, it's optimal to mount a secret as a volume

Provisioning a PersistentVolume and then populating it with the certs isn't a good workflow

Also, copying credentials such as private keys etc into a container image is not good practice

willemveerman avatar Dec 07 '23 16:12 willemveerman