dashboard icon indicating copy to clipboard operation
dashboard copied to clipboard

Fix permission denied while running docker-in-docker CI builds

Open Basavaraju-G opened this issue 3 years ago • 4 comments

Changes

CI builds are failed with permission denied while running browser e2e testcase. This commit will remove node user and use default user. https://dashboard.dogfooding.tekton.dev/#/namespaces/bastion-z/pipelineruns/tekton-dashboard-s390x-nightly-run-np27x?pipelineTask=e2e-test-dashboard&step=run-e2e-tests&retry=0

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

See the contribution guide for more details.

Basavaraju-G avatar Oct 14 '22 05:10 Basavaraju-G

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: To complete the pull request process, please assign briangleeson after the PR has been reviewed. You can assign the PR to them by writing /assign @briangleeson in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

tekton-robot avatar Oct 14 '22 05:10 tekton-robot

/retest

Basavaraju-G avatar Oct 14 '22 05:10 Basavaraju-G

/assign @AlanGreene

Basavaraju-G avatar Oct 14 '22 05:10 Basavaraju-G

@AlanGreene would you please review this PR?

Basavaraju-G avatar Oct 14 '22 06:10 Basavaraju-G

@Basavaraju-G The logs for the run you linked are no longer available. Can you confirm whether the error you're seeing is similar to the following or something different?

Logs from today's run: https://dashboard.dogfooding.tekton.dev/#/namespaces/bastion-z/pipelineruns/tekton-dashboard-s390x-nightly-run-kfcqd?pipelineTask=e2e-test-dashboard&step=run-e2e-tests&retry=0

Warning: We failed to record the video.

This error will not alter the exit code.

Error: ffmpeg exited with code 1: /home/node/cypress/videos/about.cy.js.mp4: Permission denied

    at ChildProcess.<anonymous> (/home/node/.cache/Cypress/10.9.0/Cypress/resources/app/node_modules/fluent-ffmpeg/lib/processor.js:182:22)
    at ChildProcess.emit (node:events:526:28)
    at ChildProcess.emit (node:domain:475:12)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)

Warning: We failed processing this video.

This error will not alter the exit code.

Error: ffmpeg exited with code 1: /home/node/cypress/videos/about.cy.js.mp4: Permission denied

    at ChildProcess.<anonymous> (/home/node/.cache/Cypress/10.9.0/Cypress/resources/app/node_modules/fluent-ffmpeg/lib/processor.js:182:22)
    at ChildProcess.emit (node:events:526:28)
    at ChildProcess.emit (node:domain:475:12)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)

Deleting video for passing test
An error was thrown in your plugins file while executing the handler for the after:spec event.

The error we received was:

Error: ENOENT: no such file or directory, unlink '/home/node/cypress/videos/about.cy.js.mp4'
ERROR: Browser E2E tests failed
==================================
==== INTEGRATION TESTS FAILED ====
==================================

The e2e tests running in prow are already run in a docker-in-docker environment, so this is likely something specific to your setup for the tests on Z.

We do not want to run the tests as root which is why we explicitly switch to the provided node user.

Since you haven't provided a path in the ARTIFACTS environment variable the test script creates a temporary directory to store the videos, you can see this output in the logs: Videos of failing tests will be stored at /tmp/tmp.bIBjSXIiph/videos which must be writeable.

Are you planning to store the recorded videos of the tests? If not we could provide a way to disable the recording and avoid these errors entirely. If you do want to store the videos you should provide a path in the ARTIFACTS environment variable to the location you want, and the user running the step should have permission to write to this folder.

AlanGreene avatar Oct 16 '22 14:10 AlanGreene

PR open to make recording of videos opt-in. If the ARTIFACTS variable is not set the recording of videos will be skipped.

#2536

AlanGreene avatar Oct 16 '22 23:10 AlanGreene

@AlanGreene yes im getting same error. will try again, after your PR merges https://github.com/tektoncd/dashboard/pull/2536 Screenshot 2022-10-17 at 8 59 27 AM

Basavaraju-G avatar Oct 17 '22 03:10 Basavaraju-G

Same error on P will get resolved after PR #2536

kabhiibm avatar Oct 17 '22 08:10 kabhiibm

#2536 is merged, can you try running your pipelines again?

AlanGreene avatar Oct 17 '22 10:10 AlanGreene

@Basavaraju-G: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot avatar Oct 17 '22 10:10 tekton-robot

#2536 is merged, can you try running your pipelines again?

okay. thanks

Basavaraju-G avatar Oct 18 '22 03:10 Basavaraju-G

@Basavaraju-G Looking at recent runs in the bastion-z namespace I see the update mentioned above didn't work as expected 😞

However, I found the cause. There's a script in tektoncd/plumbing that's being loaded and sets ARTIFACTS to a temp folder if the variable is not already set. This means we cannot use the presence of that variable as our flag as I had originally hoped, instead we need a separate variable we can use to explicitly disable recording videos.

https://github.com/tektoncd/dashboard/pull/2541 introduces support for setting E2E_VIDEO=false to disable recording. Once that PR is merged and your pipelines have been updated to set this environment variable we should be in business 🤞


UPDATE: #2541 is merged so you should be able to set E2E_VIDEO=false in your pipelines to skip this now.

AlanGreene avatar Oct 19 '22 13:10 AlanGreene

PR to use video-skip flag in plumbing repo: https://github.com/tektoncd/plumbing/pull/1248

Basavaraju-G avatar Oct 20 '22 06:10 Basavaraju-G