karmada icon indicating copy to clipboard operation
karmada copied to clipboard

e2e suspend: ensure proper cleanup after each run

Open a7i opened this issue 1 year ago • 12 comments

What type of PR is this? /kidn failing-test

What this PR does / why we need it: ClusterRole and CPP are cluster-level resources so we need to ensure that we use a unique name. Otherwise, the randomness is bound to have collisions here and there.

Which issue(s) this PR fixes: Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


a7i avatar Aug 28 '24 03:08 a7i

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign xishanyongye-chang for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

karmada-bot avatar Aug 28 '24 03:08 karmada-bot

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 31.47%. Comparing base (d16f098) to head (89b38f7). Report is 38 commits behind head on master.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5440      +/-   ##
==========================================
+ Coverage   30.82%   31.47%   +0.65%     
==========================================
  Files         640      643       +3     
  Lines       44414    44436      +22     
==========================================
+ Hits        13690    13987     +297     
+ Misses      29744    29423     -321     
- Partials      980     1026      +46     
Flag Coverage Δ
unittests 31.47% <ø> (+0.65%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Aug 28 '24 03:08 codecov-commenter

@a7i Glad to see you working on this fix. Could you describe the reasons for the intermittent failures in the e2e tests?

zhzhuang-zju avatar Aug 28 '24 07:08 zhzhuang-zju

hi here is some problem with lint ci~

besides, I create a issue to record recent failure in #5443

we can work together to solve it.

chaosi-zju avatar Aug 28 '24 08:08 chaosi-zju

I don't think this worked as I see a failure: https://github.com/a7i/karmada/actions/runs/10606048224/job/29396128326 😭

a7i avatar Aug 29 '24 00:08 a7i

Spent some time trying to understand why the error would occur sporadically, but failed and everything seems to be working.

XiShanYongYe-Chang avatar Aug 29 '24 12:08 XiShanYongYe-Chang

Looks good, going to try one more time

/test ?

a7i avatar Aug 30 '24 02:08 a7i

@a7i: No presubmit jobs available for karmada-io/karmada@master

In response to this:

Looks good, going to try one more time

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

karmada-bot avatar Aug 30 '24 02:08 karmada-bot

/retest e2e

a7i avatar Aug 30 '24 02:08 a7i

@a7i: No presubmit jobs available for karmada-io/karmada@master

In response to this:

/retest e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

karmada-bot avatar Aug 30 '24 02:08 karmada-bot

You need to push the code again. The retest only runs the failed CI.

XiShanYongYe-Chang avatar Aug 30 '24 03:08 XiShanYongYe-Chang

Sounds good! Will be doing a few force pushes to test

a7i avatar Aug 30 '24 14:08 a7i

Screenshot 2024-08-30 at 3 05 34 PM

one more push coming...

a7i avatar Aug 30 '24 19:08 a7i

totally unrelated test failed so perhaps that needs clean up as well

[Delete] clusterPropagation testing delete clusterPropagation and remove the labels and annotations from the resource template and reference clusterBinding [It] delete the old ClusterPropagationPolicy to unbind and create a new one

a7i avatar Aug 30 '24 20:08 a7i

It should have something to do with the new wating.

• [FAILED] [300.224 seconds]
[Suspend] clusterPropagation testing suspend the ClusterPropagationPolicy dispatching [It] adds suspend dispatching condition to Work
/home/runner/work/karmada/karmada/test/e2e/clusterpropagationpolicy_test.go:1089

  Captured StdOut/StdErr Output >>
  I0830 21:50:51.234923   57576 rbac.go:319] Waiting for clusterRoleBinding(system:test-suspend-clusterrole-mtfn4) disappear on cluster(member1)
  << Captured StdOut/StdErr Output

  Timeline >>
  STEP: Creating ClusterPropagationPolicy(clusterrole-mtfn4) @ 08/30/24 21:45:51.02
  STEP: Creating ClusterRole(system:test-suspend-clusterrole-mtfn4) @ 08/30/24 21:45:51.052
  STEP: Updating ClusterPropagationPolicy(clusterrole-mtfn4) spec @ 08/30/24 21:45:51.059
  [FAILED] in [It] - /home/runner/work/karmada/karmada/test/e2e/clusterpropagationpolicy_test.go:1097 @ 08/30/24 21:50:51.108
  STEP: Removing ClusterPropagationPolicy(clusterrole-mtfn4) @ 08/30/24 21:50:51.222
  STEP: Remove ClusterRole(system:test-suspend-clusterrole-mtfn4) @ 08/30/24 21:50:51.228
  << Timeline

  [FAILED] Timed out after 300.001s.
  Expected
      <bool>: false
  to equal
      <bool>: true
  In [It] at: /home/runner/work/karmada/karmada/test/e2e/clusterpropagationpolicy_test.go:1097 @ 08/30/24 21:50:51.108

  Full Stack Trace
    github.com/karmada-io/karmada/test/e2e.init.func7.4.3()
    	/home/runner/work/karmada/karmada/test/e2e/clusterpropagationpolicy_test.go:1097 +0x288

XiShanYongYe-Chang avatar Aug 31 '24 01:08 XiShanYongYe-Chang

/ok-to-test /retest

liangyuanpeng avatar Sep 03 '24 14:09 liangyuanpeng

closing in favor of #5525

a7i avatar Sep 11 '24 14:09 a7i