cloud-provider-gcp icon indicating copy to clipboard operation
cloud-provider-gcp copied to clipboard

[WIP] Initial e2e test migration from in-tree to gcp cloud provider

Open seans3 opened this issue 1 year ago • 4 comments

  • Initial migration of in-tree gce e2e tests to out-of-tree cloud-provider-gcp repository.
    • Currently only three tests: 1) A simple firewall test, 2) a load-balancer (GCE-specific) test, and 3) a simple node test.
    • Tests 1 and 3 currently pass, but inclusion of the GCE-specific test is causing issues.
  • Migrate the gce provider code for the e2e tests.
  • Includes script tools/run-e2e-test.sh to run the e2e tests from the command line.
    • Builds the e2e.test binary by running go test -c within the test/e2e directory.
    • Runs the e2e test using the gce provider and the ginkgo tester.

Current Status

  • The e2e test binary builds using the gce provider code locally.
  • The command-line e2e test correctly starts the cluster
  • The command-line e2e test starts the ginkgo tester against the cluster
  • Basic e2e tests not calling the gce provider functions run successfully
  • Currently the e2e test runs the ginkgo tester with the following output:
⚙️ /home/sean/go/bin/kubetest2-tester-ginkgo --use-built-binaries true --parallel=30 --test-args=--provider=gce --gce-project=seans-devel --gce-zone=us-central1-c --minStartupPods=8 --focus-regex=\[cloud-provider-gcp-e2e\]
I0505 17:39:38.713100  361989 ginkgo.go:131] Using kubeconfig at /home/sean/go/src/k8s.io/cloud-provider-gcp/_rundir/sean-test/kubetest2-kubeconfig
I0505 17:39:38.713164  361989 ginkgo.go:100] Running ginkgo test as /home/sean/go/src/k8s.io/cloud-provider-gcp/_rundir/sean-test/ginkgo [--nodes=30 /home/sean/go/src/k8s.io/cloud-provider-gcp/_rundir/sean-test/e2e.test -- --kubeconfig=/home/sean/go/src/k8s.io/cloud-provider-gcp/_rundir/sean-test/kubetest2-kubeconfig --kubectl-path=/home/sean/go/src/k8s.io/cloud-provider-gcp/_rundir/sean-test/kubectl --ginkgo.skip= --ginkgo.focus=\[cloud-provider-gcp-e2e\] --report-dir=/home/sean/go/src/k8s.io/cloud-provider-gcp/_artifacts --ginkgo.timeout=24h0m0s --provider=gce --gce-project=seans-devel --gce-zone=us-central1-c --minStartupPods=8]
  • Current e2e tests calling into the gce provider have the following auth error:
 I0505 17:39:38.768046 362206 gce.go:48] Fetching cloud provider for "gce"
 E0505 17:39:38.772713  362206 test_context.go:581] Failed to setup provider config for "gce": Error building GCE/GKE provider: google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for more information

seans3 avatar May 06 '24 01:05 seans3

This issue is currently awaiting triage.

If the repository mantainers determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar May 06 '24 01:05 k8s-ci-robot

/assign @BenTheElder /assign @aojea

seans3 avatar May 06 '24 01:05 seans3

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: seans3 Once this PR has been reviewed and has the lgtm label, please ask for approval from aojea. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar May 06 '24 01:05 k8s-ci-robot

First pass: this is generally looking good, we'll want to stand up a job to actually run this in CI (we can make that manually run at first) and sort out what to do re: Bazel.

BenTheElder avatar May 07 '24 21:05 BenTheElder

First pass: this is generally looking good, we'll want to stand up a job to actually run this in CI (we can make that manually run at first) and sort out what to do re: Bazel.

yeah https://github.com/kubernetes/test-infra/pull/32603

aojea avatar May 13 '24 10:05 aojea

/test pull-cloud-provider-gcp-e2e

aojea avatar May 13 '24 11:05 aojea

@seans3: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cloud-provider-gcp-e2e beb6614a3acce559f33fcaed01d41d8ecf3cfa2f link false /test pull-cloud-provider-gcp-e2e

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

k8s-ci-robot avatar May 13 '24 11:05 k8s-ci-robot

/home/prow/go/src/cloud-provider-gcp
cp: cannot stat '/root/go/bin/ginkgo': No such file or directory
+ EXIT_VALUE=1
+ set +o xtrace
Cleaning up after docker in docker.

aojea avatar May 13 '24 15:05 aojea

/test pull-cloud-provider-gcp-e2e

seans3 avatar May 18 '24 03:05 seans3

/test pull-cloud-provider-gcp-e2e

seans3 avatar May 18 '24 05:05 seans3

/test pull-cloud-provider-gcp-e2e

seans3 avatar May 18 '24 06:05 seans3

/test pull-cloud-provider-gcp-e2e

seans3 avatar May 18 '24 20:05 seans3

/test pull-cloud-provider-gcp-e2e

seans3 avatar May 21 '24 04:05 seans3

/test pull-cloud-provider-gcp-e2e

seans3 avatar May 21 '24 05:05 seans3

it went through 👏

aojea avatar May 21 '24 15:05 aojea

/test pull-cloud-provider-gcp-e2e

seans3 avatar May 21 '24 17:05 seans3

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/cloud-provider-gcp/683/pull-cloud-provider-gcp-e2e/1792967087560331264

sweet!

BenTheElder avatar May 21 '24 20:05 BenTheElder

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder, seans3

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar May 21 '24 20:05 k8s-ci-robot

/test pull-cloud-provider-gcp-e2e

seans3 avatar May 21 '24 21:05 seans3

/triage accepted

seans3 avatar May 21 '24 22:05 seans3

/hold cancel

seans3 avatar May 21 '24 22:05 seans3

/lgtm

BenTheElder avatar May 21 '24 22:05 BenTheElder