Unable to copy parameter files from bitbucket to Kubernetes cluster through testkube create test open API

[chandana194]

Hi I am unable copy parameter files and script files from bitbucket to Kubernetes where testkube is running.

Could you please let me know how can I achieve this through create new test API

[vLia]

Hello @chandana194,

Could you please provide some more information:

• what version of Testkube are you using?
• what commands did you use to run the test?
• what errors are you getting?

You can use kubectl testkube debug info to get these details

[chandana194]

CLIENT VERSION | SERVER VERSION | CLUSTER VERSION
-----------------+----------------+------------------ 1.5.20 | v1.5.3 | v1.20.15

I am trying to create a test using /testkube-api-server/v1/tests endpoint by passing bitbucket details and the script path , and trying to copy few parameter files from my local to cluster. The test is getting created with 200 response but when I try to run it the testkube job is getting failed on the cluster

{ "name": "param-test2", "namespace": "testkube", "type": "k6/script", "content": { "type": "git-file", "repository": { "type": "git", "uri": "https://tools.lowes.com/path_to_git.git", "branch": "main", "commit": "822eabd78a1", "path": "", "username": "my_cred", "token": "valid token is passed"
}, "data": "testJSON.js", "uri": "/stash/projects/path-to-file/testJSON.js" }, "source": "DACI_ScoreCard_PerfTesting", "created": "2022-07-30T06:54:15Z",

"copyFiles": [ "/Users/cnagara/Documents/K6Scripts/ParamsInK6/data.json:/data/data.json" ], "executionRequest": { "name": "testing parameters option", "testSuiteName": "", "number": 0, "executionLabels": { "users": "3", "prefix": "some-" }, "namespace": "testkube",

"image": "kubeshop/testkube-executor-custom:1.10.11-dev-0a9c91",
"envs": {
  "record": "true",
  "prefix": "some-"
},

"sync": true,
"copyFiles": {
  "property1": [
    null
  ],
  "property2": [
    null
  ]
}

} }

[vLia]

Testkube is at version 1.5.38 at the moment, files were introduced at 1.5.34. Could you upgrade your local installation and share the exact errors that are thrown?

[chandana194]

Sure I will update Testkube version and share the errors

[chandana194]

Hi @vLia I tried updating testkube to latest version, Below is the issue I am facing while installing server components :

Testkube client version is [ Client Version 1.5.41] helm repo update testkube init WELCOME TO

████████ ███████ ███████ ████████ ██ ██ ██ ██ ██████ ███████ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██
██ █████ ███████ ██ █████ ██ ██ ██████ █████
██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██
██ ███████ ███████ ██ ██ ██ ██████ ██████ ███████ /tɛst kjub/ by Kubeshop

Helm installing testkube framework

Installing testkube (error: process error: exit status 1 output: Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition )

On the minikube k8s cluster I see below issue for the pod [webhook-cert-create-qqrkx] Failed to pull image "bitnami/kubectl:1.5.8": rpc error: code = Unknown desc = Error response from daemon: manifest for bitnami/kubectl:1.5.8 not found: manifest unknown: manifest unknown

Could you please let me know how can I fix it ?

[vLia]

Hello @chandana194, Thank you for reporting this. We are facing some difficulties in the install process right now, and are currently working on rolling back the changes that broke the process. Will let you know as soon as it works again. Thank you for your patience!

[vsukhin]

Just released a fix. Should be solved

[chandana194]

Thank you for the update @vsukhin

Hi @vLia , Have updated the testkube version and tried running the test by placing the script on bitbucket repo and passing the git hub credentials through create test API

Current version Client Version 1.5.41 Server Version v1.5.3

Still I could see that, it is unable to fetch the script from repo and the job is getting failed . there are no logs present when I use the below command shared by you kubectl testkube debug info

Below are the API server pod logs, but its is not that helpful. Could you please help me on this ?

{"level":"info","ts":1664777495.5552132,"caller":"v1/tests.go:341","msg":"creating test","request":{"name":"param-test6","namespace":"testkube","type":"k6/script","content":{"type":"git-file","repository":{"type":"git","uri":"https://tools.lowes.com/stash/scm/e-pep/daci_scorecard_perftesting.git","branch":"main","commit":"4794f5e03bc","username":"","token":"MTA4ODE1NzkzNDQ5OmlFLI9m2c9aAguNbF2tI"},"data":"testJSON.js","uri":"/stash/projects/E-PEP/repos/daci_scorecard_perftesting/browse/testJSON.js"},"created":"2022-07-30T06:54:15Z","executionRequest":{"name":"testing parameters option","executionLabels":{"prefix":"some-","users":"3"},"namespace":"testkube","image":"kubeshop/testkube-executor-custom:1.10.11-dev-0a9c91","envs":{"prefix":"some-","record":"true"},"sync":true}}} {"level":"info","ts":1664777565.1154668,"caller":"v1/executions.go:166","msg":"calling executor with options","options":{"name":"testing parameters option-1","number":1,"namespace":"testkube","envs":{"prefix":"some-","record":"true"}}} {"level":"info","ts":1664777565.1188643,"caller":"slacknotifier/slack.go:281","msg":"Execution changed","status":"running"} {"level":"warn","ts":1664777565.3895643,"caller":"slacknotifier/slack.go:223","msg":"error while getting bot channels","error":"not_authed"} {"level":"warn","ts":1664777565.3896062,"caller":"webhook/emitter.go:157","msg":"notify slack failed","error":"not_authed"} {"level":"info","ts":1664777565.9257514,"caller":"v1/executions.go:226","msg":"test executed","executionId":"633a7d5da15807833ce9d55c","status":"running"} {"level":"error","ts":1664777568.5308352,"caller":"client/job.go:272","msg":"waiting for pod complete error","executionID":"633a7d5da15807833ce9d55c","type":"async","error":"pod testkube/633a7d5da15807833ce9d55c-kzkld failed","stacktrace":"github.com/kubeshop/testkube/pkg/executor/client.JobExecutor.updateResultsFromPod\n\t/build/pkg/executor/client/job.go:272\ngithub.com/kubeshop/testkube/pkg/executor/client.JobExecutor.Execute.func1\n\t/build/pkg/executor/client/job.go:195"} {"level":"info","ts":1664777568.5473704,"caller":"client/job.go:298","msg":"execution completed saving result","executionID":"633a7d5da15807833ce9d55c","type":"async","executionId":"633a7d5da15807833ce9d55c","status":"failed"} {"level":"info","ts":1664777568.5594604,"caller":"slacknotifier/slack.go:281","msg":"Execution changed","status":"failed"} {"level":"warn","ts":1664777568.6065674,"caller":"slacknotifier/slack.go:223","msg":"error while getting bot channels","error":"not_authed"} {"level":"warn","ts":1664777568.6066043,"caller":"webhook/emitter.go:157","msg":"notify slack failed","error":"not_authed"}

[chandana194]

[chandana194]

Tried running through testkube dashboard and I could see the below error {"id":"633adc6ba15807833ce9d56f","testName":"param-check-git1","testNamespace":"testkube","testType":"k6/script","name":"param-check-git1-1","number":1,"content":{"type":"git-dir","repository":{"type":"git-dir","uri":"https://tools.lowes.com/stash/scm/e-pep/daci_scorecard_perftesting.git","branch":"master","path":"/stash/projects/E-PEP/repos/daci_scorecard_perftesting/browse/testJSON.js"}},"startTime":"2022-10-03T12:58:19.399Z","endTime":"2022-10-03T12:58:25.404Z","duration":"00:00:06","executionResult":{"status":"failed","errorMessage":"k6 test script /data/repo/run not found"}}

Please let me know , In case, If we are making any mistake here

[vLia]

Hello @chandana194, K6 tests do not support using directories, we have an open issue to reflect this in Testkube as well (https://github.com/kubeshop/testkube/issues/2400). You should aim for git-files, as you are already using the testJSON.js file. At the moment however, we are observing some errors on the K6 executor, you can monitor the progress on https://github.com/kubeshop/testkube/issues/2399 I'll let you know once that is fixed.

[chandana194]

Thank you for the update @vLia . Please let me know as soon as you have a fix for this as this feature is very critical for our app testing.

[chandana194]

Hi @vLia Meanwhile, Could you please let me know how I can copy testdata file while creating tests on testkube [ like I would like to store 100 unique usernames in a json file and access it in K6 script]

I see below error - even though I use copyFiles parameter in the CREATE test request body. Please let me know, how we can create a testdata file copy while running tests through testkube "copyFiles": [ "/Users/cnagara/Documents/K6Scripts/ParamsInK6/data.json:/data/data.json"

{"type":"line","content":"time="2022-10-03T09:22:59Z" level=error msg="GoError: stat /Users/cnagara/Documents/K6Scripts/ParamsInK6/data.json: no such file or directory\n\tat go.k6.io/k6/js.(*InitContext).Open-fm (native)\n\tat file:///data/test-content:47:25(6)\n\tat file:///data/test-content:46:11(59)\n" hint="script exception"\n"} {"type":"result","result":{"status":"failed","errorMessage":"process error: exit status 107"}}

[vLia]

Hello @chandana194, The git-file issue has been fixed.

The copy-files parameter works only with the kubectl command. The file is not stored on the test, it is read on kubectl run test and sent to the execution.

[chandana194]

Hi @vLia Thank you for the update. Any help from API side. also, to copy test data parameter files which needs to be accessed from the script.

[chandana194]

We are mainly looking for a solution to handle parameterisation in k6 through testkube. We have requirement to test an API with multiple unique data. Testkube is not able to run the test, If we keep all the test data inside the script. So it will be helpful we can pass parameter files to testkube

[chandana194]

Hello @chandana194, The git-file issue has been fixed.

The copy-files parameter works only with the kubectl command. The file is not stored on the test, it is read on kubectl run test and sent to the execution.

Still I see this error with the below version, Do I need to upgrade ? Client Version 1.5.41 Server Version v1.5.3

{"id":"6343f178a15807833ce9d79e","testName":"param-check-public-repo1","testNamespace":"testkube","testType":"k6/script","name":"param-check-public-repo1-1","number":1,"content":{"type":"git-file","repository":{"type":"git-file","uri":"https://github.com/chandana194/K6-perf-testing.git","branch":"main"}},"startTime":"2022-10-10T10:18:32.215Z","endTime":"2022-10-10T10:18:37.771Z","duration":"00:00:05","executionResult":{"status":"failed","errorMessage":"process error: exit status 255"}}

[vLia]

Hello @chandana194 , On API level you could send the content of the file on execution, as shown in here: https://kubeshop.github.io/testkube/openapi#tag/api/operation/executeTests

Could you please specify why Testkube cannot run the test if the data is in the script?

To debug why it is not working, the first step is to get the logs of the pod. On execution, Testkube creates a job which creates a pod. Do you have access to this information?

[vsukhin]

Hey @chandana194

You're almost there for running git file based k6s script. Only one parameter is missed path to your k6 script. It should be something like "path": "your_k6_directory/subdir/k6script.js"

[chandana194]

Hello @chandana194 , On API level you could send the content of the file on execution, as shown in here: https://kubeshop.github.io/testkube/openapi#tag/api/operation/executeTests

Could you please specify why Testkube cannot run the test if the data is in the script?

To debug why it is not working, the first step is to get the logs of the pod. On execution, Testkube creates a job which creates a pod. Do you have access to this information?

We are trying to send 10K data through array as below , Test gets created without any issues but when we try to run it - the tests gets failed and there are no logs present on the pod/job . Screenshots are attached for reference.

[chandana194]

able to run the same with K6 but unable to run through testkube. So would like to know is there any way we can store these test data in a file and pass it to testkube externally

[vLia]

File sizes currently have a low limit. We discussed it in the team, and I am creating an issue to address this.

[vLia]

closing this, tracking it in https://github.com/kubeshop/testkube/issues/2501