hyperconverged-cluster-operator icon indicating copy to clipboard operation
hyperconverged-cluster-operator copied to clipboard

Introduce cluster level EvictionStrategy

Open acardace opened this issue 2 years ago • 49 comments

use LiveMigrate as a default value and a couple of unit tests.

Signed-off-by: Antonio Cardace [email protected]

Reviewer Checklist

Reviewers are supposed to review the PR for every aspect below one by one. To check an item means the PR is either "OK" or "Not Applicable" in terms of that item. All items are supposed to be checked before merging a PR.

  • [ ] PR Message
  • [ ] Commit Messages
  • [ ] How to test
  • [ ] Unit Tests
  • [ ] Functional Tests
  • [ ] User Documentation
  • [ ] Developer Documentation
  • [ ] Upgrade Scenario
  • [ ] Uninstallation Scenario
  • [ ] Backward Compatibility
  • [ ] Troubleshooting Friendly

Release note:

Introduce cluster level EvictionStrategy field.

acardace avatar Aug 31 '22 11:08 acardace

/cc @tiraboschi

acardace avatar Aug 31 '22 11:08 acardace

Pull Request Test Coverage Report for Build 5043198110

  • 1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.002%) to 85.817%

Totals Coverage Status
Change from base Build 5003203485: 0.002%
Covered Lines: 4907
Relevant Lines: 5718

💛 - Coveralls

coveralls avatar Aug 31 '22 11:08 coveralls

@tiraboschi can someone take a look at this?

acardace avatar Sep 06 '22 12:09 acardace

@acardace should this match Kubevirt values? https://github.com/kubevirt/kubevirt/blob/16ca54fc6e4382a78ff315cc873cc692cdd1d62d/staging/src/kubevirt.io/api/core/v1/types.go#L1875-L1879

tiraboschi avatar Sep 06 '22 16:09 tiraboschi

@acardace should this match Kubevirt values? https://github.com/kubevirt/kubevirt/blob/16ca54fc6e4382a78ff315cc873cc692cdd1d62d/staging/src/kubevirt.io/api/core/v1/types.go#L1875-L1879

Yes, the issue with functional tests comes from this. We are getting

Internal error occurred: failed calling webhook "virtualmachineinstances-create-validator.kubevirt.io": failed to call webhook: Post "[https://virt-api.kubevirt-hyperconverged.svc:443/virtualmachineinstances-validate-create?timeout=10s](https://virt-api.kubevirt-hyperconverged.svc/virtualmachineinstances-validate-create?timeout=10s)": EOF

and indeed in https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/kubevirt_hyperconverged-cluster-operator/2071/pull-ci-kubevirt-hyperconverged-cluster-operator-main-hco-e2e-image-index-aws/1567189626756534272/artifacts/hco-e2e-image-index-aws/e2e-test/artifacts/kubevirt-must-gather/quay-io-kubevirt-must-gather-sha256-675cac0b2d2ef3ef3dcce5e16d75cae174a16074499c3253fda184c465ce2746/namespaces/kubevirt-hyperconverged/pods/virt-api-7fc64dfb68-ndn8x/virt-api/virt-api/logs/current.log

we see:

2022-09-06T18:57:29.243345170Z {"component":"virt-api","level":"info","msg":"http: panic serving 10.128.0.1:38264: runtime error: invalid memory address or nil pointer dereference\ngoroutine 3597 [running]:\nnet/http.(*conn).serve.func1()\n\tGOROOT/src/net/http/server.go:1802 +0xb9\npanic({0x1d2dc60, 0x382a5e0})\n\tGOROOT/src/runtime/panic.go:1047 +0x266\nkubevirt.io/kubevirt/pkg/virt-api/webhooks/validating-webhook/admitters.validateLiveMigration(0xc0088685a0, 0xc006802798, 0x0)\n\tpkg/virt-api/webhooks/validating-webhook/admitters/vmi-create-admitter.go:779 +0x145\nkubevirt.io/kubevirt/pkg/virt-api/webhooks/validating-webhook/admitters.ValidateVirtualMachineInstanceSpec(0x1fd174e, 0xc006802798, 0x0)\n\tpkg/virt-api/webhooks/validating-webhook/admitters/vmi-create-admitter.go:208 +0x1d4b\nkubevirt.io/kubevirt/pkg/virt-api/webhooks/validating-webhook/admitters.(*VMICreateAdmitter).Admit(0xc0004b11c0, 0xc008779c50)\n\tpkg/virt-api/webhooks/validating-webhook/admitters/vmi-create-admitter.go:100 +0x10c\nkubevirt.io/kubevirt/pkg/util/webhooks/validating-webhooks.Serve({0x2507758, 0xc008716ee0}, 0xc008716f20, {0x24ccc60, 0xc0004b11c0})\n\tpkg/util/webhooks/validating-webhooks/validating-webhook.go:73 +0xc9\nkubevirt.io/kubevirt/pkg/virt-api/webhooks/validating-webhook.ServeVMICreate(...)\n\tpkg/virt-api/webhooks/validating-webhook/validating-webhook.go:34\nkubevirt.io/kubevirt/pkg/virt-api.(*virtAPIApp).registerValidatingWebhooks.func1({0x2507758, 0xc008716ee0}, 0x0)\n\tpkg/virt-api/api.go:773 +0x7a\nnet/http.HandlerFunc.ServeHTTP(0x0, {0x2507758, 0xc008716ee0}, 0xc008716ee0)\n\tGOROOT/src/net/http/server.go:2047 +0x2f\nnet/http.(*ServeMux).ServeHTTP(0xc0086e379e, {0x2507758, 0xc008716ee0}, 0xc00846b600)\n\tGOROOT/src/net/http/server.go:2425 +0x149\nnet/http.serverHandler.ServeHTTP({0x24f8900}, {0x2507758, 0xc008716ee0}, 0xc00846b600)\n\tGOROOT/src/net/http/server.go:2879 +0x43b\nnet/http.(*conn).serve(0xc0069e2500, {0x250a218, 0xc00261af00})\n\tGOROOT/src/net/http/server.go:1930 +0xb08\ncreated by net/http.(*Server).Serve\n\tGOROOT/src/net/http/server.go:3034 +0x4e8\n","pos":"server.go:3160","timestamp":"2022-09-06T18:57:29.243292Z"}

So yes, we are passing a bad value here but "virtualmachineinstances-create-validator.kubevirt.io" should definitively be more resilient on bad configurations.

tiraboschi avatar Sep 07 '22 06:09 tiraboschi

@tiraboschi do you understand why all the index tests are failing?

acardace avatar Sep 07 '22 11:09 acardace

@tiraboschi do you understand why all the index tests are failing?

They are failing here on the test for the default value:

++ oc get hco -n kubevirt-hyperconverged kubevirt-hyperconverged -o 'jsonpath={.spec.evictionStrategy}'
+ EVICTION_STRATEGY=LiveMigrate
+ [[ EvictionStrategyLiveMigrate != \L\i\v\e\M\i\g\r\a\t\e ]]
+ echo 'Failed checking CR defaults for evictionStrategy'

please fix the expected value on line 33 of hack/check_defaults.sh

tiraboschi avatar Sep 07 '22 11:09 tiraboschi

@tiraboschi do you understand why all the index tests are failing?

They are failing here on the test for the default value:

++ oc get hco -n kubevirt-hyperconverged kubevirt-hyperconverged -o 'jsonpath={.spec.evictionStrategy}'
+ EVICTION_STRATEGY=LiveMigrate
+ [[ EvictionStrategyLiveMigrate != \L\i\v\e\M\i\g\r\a\t\e ]]
+ echo 'Failed checking CR defaults for evictionStrategy'

please fix the expected value on line 33 of hack/check_defaults.sh

I Fixed that, thank you.

acardace avatar Sep 07 '22 13:09 acardace

hco-e2e-upgrade-index-sno-aws lane succeeded. /override ci/prow/hco-e2e-upgrade-index-sno-azure

hco-bot avatar Sep 07 '22 15:09 hco-bot

@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-upgrade-index-sno-azure

In response to this:

hco-e2e-upgrade-index-sno-aws lane succeeded. /override ci/prow/hco-e2e-upgrade-index-sno-azure

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot avatar Sep 07 '22 15:09 kubevirt-bot

hco-e2e-upgrade-prev-index-sno-aws lane succeeded. /override ci/prow/hco-e2e-upgrade-prev-index-sno-azure okd-hco-e2e-upgrade-index-aws lane succeeded. /override ci/prow/okd-hco-e2e-upgrade-index-gcp hco-e2e-upgrade-index-aws lane succeeded. /override ci/prow/hco-e2e-upgrade-index-azure

hco-bot avatar Sep 07 '22 15:09 hco-bot

@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-upgrade-index-azure, ci/prow/hco-e2e-upgrade-prev-index-sno-azure, ci/prow/okd-hco-e2e-upgrade-index-gcp

In response to this:

hco-e2e-upgrade-prev-index-sno-aws lane succeeded. /override ci/prow/hco-e2e-upgrade-prev-index-sno-azure okd-hco-e2e-upgrade-index-aws lane succeeded. /override ci/prow/okd-hco-e2e-upgrade-index-gcp hco-e2e-upgrade-index-aws lane succeeded. /override ci/prow/hco-e2e-upgrade-index-azure

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot avatar Sep 07 '22 15:09 kubevirt-bot

okd-hco-e2e-image-index-aws lane succeeded. /override ci/prow/okd-hco-e2e-image-index-gcp

hco-bot avatar Sep 07 '22 15:09 hco-bot

@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/okd-hco-e2e-image-index-gcp

In response to this:

okd-hco-e2e-image-index-aws lane succeeded. /override ci/prow/okd-hco-e2e-image-index-gcp

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot avatar Sep 07 '22 15:09 kubevirt-bot

hco-e2e-image-index-gcp lane succeeded. /override ci/prow/hco-e2e-image-index-azure hco-e2e-image-index-gcp lane succeeded. /override ci/prow/hco-e2e-image-index-aws

hco-bot avatar Sep 07 '22 16:09 hco-bot

@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-image-index-aws, ci/prow/hco-e2e-image-index-azure

In response to this:

hco-e2e-image-index-gcp lane succeeded. /override ci/prow/hco-e2e-image-index-azure hco-e2e-image-index-gcp lane succeeded. /override ci/prow/hco-e2e-image-index-aws

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot avatar Sep 07 '22 16:09 kubevirt-bot

hco-e2e-kv-smoke-gcp lane succeeded. /override ci/prow/hco-e2e-kv-smoke-azure

hco-bot avatar Sep 07 '22 17:09 hco-bot

@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-kv-smoke-azure

In response to this:

hco-e2e-kv-smoke-gcp lane succeeded. /override ci/prow/hco-e2e-kv-smoke-azure

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot avatar Sep 07 '22 17:09 kubevirt-bot

/retest

tiraboschi avatar Sep 07 '22 21:09 tiraboschi

hco-e2e-upgrade-prev-index-aws lane succeeded. /override ci/prow/hco-e2e-upgrade-prev-index-azure

hco-bot avatar Sep 07 '22 23:09 hco-bot

@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-upgrade-prev-index-azure

In response to this:

hco-e2e-upgrade-prev-index-aws lane succeeded. /override ci/prow/hco-e2e-upgrade-prev-index-azure

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot avatar Sep 07 '22 23:09 kubevirt-bot

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

sonarqubecloud[bot] avatar Sep 08 '22 07:09 sonarqubecloud[bot]

hco-e2e-upgrade-index-aws lane succeeded. /override ci/prow/hco-e2e-upgrade-index-azure okd-hco-e2e-image-index-aws lane succeeded. /override ci/prow/okd-hco-e2e-image-index-gcp hco-e2e-image-index-sno-aws lane succeeded. /override ci/prow/hco-e2e-image-index-sno-azure

hco-bot avatar Sep 08 '22 09:09 hco-bot

@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-image-index-sno-azure, ci/prow/hco-e2e-upgrade-index-azure, ci/prow/okd-hco-e2e-image-index-gcp

In response to this:

hco-e2e-upgrade-index-aws lane succeeded. /override ci/prow/hco-e2e-upgrade-index-azure okd-hco-e2e-image-index-aws lane succeeded. /override ci/prow/okd-hco-e2e-image-index-gcp hco-e2e-image-index-sno-aws lane succeeded. /override ci/prow/hco-e2e-image-index-sno-azure

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot avatar Sep 08 '22 09:09 kubevirt-bot

/test okd-hco-e2e-upgrade-index-aws /test okd-hco-e2e-upgrade-index-gcp

tiraboschi avatar Sep 08 '22 10:09 tiraboschi

@tiraboschi: The specified target(s) for /test were not found. The following commands are available to trigger required jobs:

  • /test build-hco-test-utils-image
  • /test pull-hyperconverged-cluster-operator-e2e-k8s-1.23

Use /test all to run the following jobs that were automatically triggered:

  • pull-hyperconverged-cluster-operator-e2e-k8s-1.23

In response to this:

/test okd-hco-e2e-upgrade-index-aws /test okd-hco-e2e-upgrade-index-gcp

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot avatar Sep 08 '22 10:09 kubevirt-bot

/approve

tiraboschi avatar Sep 08 '22 10:09 tiraboschi

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tiraboschi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

kubevirt-bot avatar Sep 08 '22 10:09 kubevirt-bot

okd-hco-e2e-upgrade-index-aws lane succeeded. /override ci/prow/okd-hco-e2e-upgrade-index-gcp

hco-bot avatar Sep 08 '22 11:09 hco-bot

@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/okd-hco-e2e-upgrade-index-gcp

In response to this:

okd-hco-e2e-upgrade-index-aws lane succeeded. /override ci/prow/okd-hco-e2e-upgrade-index-gcp

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot avatar Sep 08 '22 11:09 kubevirt-bot