cluster-api-provider-vsphere icon indicating copy to clipboard operation
cluster-api-provider-vsphere copied to clipboard

🌱 hack/e2e don't add binary files to artifacts and also censor base64 encoded values

Open chrischdi opened this issue 1 year ago • 22 comments
trafficstars

What this PR does / why we need it:

Improves our censoring in e2e.sh to:

  • not add binary files (e.g. gzipped files) to the output artifacts because we can't control their data
  • also censor the base64 version of our secret

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #

chrischdi avatar May 13 '24 12:05 chrischdi

/test pull-cluster-api-provider-vsphere-e2e-govmomi-main

chrischdi avatar May 13 '24 14:05 chrischdi

/test pull-cluster-api-provider-vsphere-e2e-govmomi-main

/test pull-cluster-api-provider-vsphere-e2e-supervisor-main

chrischdi avatar May 13 '24 14:05 chrischdi

/test pull-cluster-api-provider-vsphere-e2e-govmomi-main

/test pull-cluster-api-provider-vsphere-e2e-supervisor-main

chrischdi avatar May 13 '24 15:05 chrischdi

It works!

Deleting file /tmp/tmp.Yr4QamCpv6/junit.e2e_suite.1.xml of type  text/xml
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/bootstrap/logs/capi-ipam-in-cluster-system/capi-ipam-in-cluster-controller-manager/capi-ipam-in-cluster-controller-manager-674c86d87d-6chkk/manager-log-metadata.json of type  application/json
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/bootstrap/logs/capi-kubeadm-control-plane-system/capi-kubeadm-control-plane-controller-manager/capi-kubeadm-control-plane-controller-manager-65f757b68b-j2rs4/manager-log-metadata.json of type  application/json
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/bootstrap/logs/capi-kubeadm-bootstrap-system/capi-kubeadm-bootstrap-controller-manager/capi-kubeadm-bootstrap-controller-manager-67847d9597-bfs25/manager-log-metadata.json of type  application/json
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/bootstrap/logs/capi-system/capi-controller-manager/capi-controller-manager-58546c9b5c-kbxbt/manager-log-metadata.json of type  application/json
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/bootstrap/logs/capv-system/capv-controller-manager/capv-controller-manager-99489f744-sgfst/manager-log-metadata.json of type  application/json
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/quick-start-ror2gi/machines/quick-start-ror2gi-cp-nbg9j-vjdnj/cloud-init.log of type  inode/x-empty
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/quick-start-ror2gi/machines/quick-start-ror2gi-md-md-0-n7wsj-5t7s2-j8fsq/cloud-init.log of type  inode/x-empty
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/quick-start-hnzumg/machines/quick-start-hnzumg-md-0-g4x6r-hjmqg/cloud-init.log of type  inode/x-empty
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/quick-start-hnzumg/machines/quick-start-hnzumg-md-0-g4x6r-hjmqg/containerd.log of type  inode/x-empty
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/quick-start-hnzumg/machines/quick-start-hnzumg-md-0-g4x6r-hjmqg/cloud-init-output.log of type  inode/x-empty
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/quick-start-hnzumg/machines/quick-start-hnzumg-md-0-g4x6r-hjmqg/kubelet.log of type  inode/x-empty
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/quick-start-hnzumg/machines/quick-start-hnzumg-vspht/cloud-init.log of type  inode/x-empty
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/quick-start-hnzumg/machines/quick-start-hnzumg-vspht/containerd.log of type  inode/x-empty
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/quick-start-hnzumg/machines/quick-start-hnzumg-vspht/cloud-init-output.log of type  inode/x-empty
Deleting file /tmp/tmp.Yr4QamCpv6/clusters/quick-start-hnzumg/machines/quick-start-hnzumg-vspht/kubelet.log of type  inode/x-empty

Changed to allow these types :-) they all seem to make sense.

Also keeping inode/x-empty so we see that these files got no content instead of not finding them anymore.

chrischdi avatar May 14 '24 11:05 chrischdi

/test ?

sbueringer avatar May 14 '24 11:05 sbueringer

@sbueringer: The following commands are available to trigger required jobs:

  • /test pull-cluster-api-provider-vsphere-e2e-govmomi-blocking-main
  • /test pull-cluster-api-provider-vsphere-e2e-govmomi-conformance-ci-latest-main
  • /test pull-cluster-api-provider-vsphere-e2e-govmomi-conformance-main
  • /test pull-cluster-api-provider-vsphere-e2e-govmomi-main
  • /test pull-cluster-api-provider-vsphere-e2e-govmomi-upgrade-1-29-1-30-main
  • /test pull-cluster-api-provider-vsphere-e2e-supervisor-blocking-main
  • /test pull-cluster-api-provider-vsphere-e2e-supervisor-conformance-ci-latest-main
  • /test pull-cluster-api-provider-vsphere-e2e-supervisor-conformance-main
  • /test pull-cluster-api-provider-vsphere-e2e-supervisor-main
  • /test pull-cluster-api-provider-vsphere-e2e-supervisor-upgrade-1-29-1-30-main
  • /test pull-cluster-api-provider-vsphere-e2e-vcsim-govmomi-main
  • /test pull-cluster-api-provider-vsphere-e2e-vcsim-supervisor-main
  • /test pull-cluster-api-provider-vsphere-test-main
  • /test pull-cluster-api-provider-vsphere-verify-main

The following commands are available to trigger optional jobs:

  • /test pull-cluster-api-provider-vsphere-apidiff-main

Use /test all to run the following jobs that were automatically triggered:

  • pull-cluster-api-provider-vsphere-apidiff-main
  • pull-cluster-api-provider-vsphere-e2e-govmomi-blocking-main
  • pull-cluster-api-provider-vsphere-e2e-supervisor-blocking-main
  • pull-cluster-api-provider-vsphere-test-main
  • pull-cluster-api-provider-vsphere-verify-main

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar May 14 '24 11:05 k8s-ci-robot

/test pull-cluster-api-provider-vsphere-e2e-govmomi-main

(just want to run it with more data, to see if any files are deleted)

sbueringer avatar May 14 '24 11:05 sbueringer

/test pull-cluster-api-provider-vsphere-e2e-supervisor-main

chrischdi avatar May 14 '24 12:05 chrischdi

Both green without any new files which got removed.

chrischdi avatar May 14 '24 13:05 chrischdi

/lgtm /approve

/hold @chrischdi We don't even run "Cleaning up VSPHERE_PASSWORD" anymore? (couldn't find it in the logs)

sbueringer avatar May 14 '24 13:05 sbueringer

LGTM label has been added.

Git tree hash: 4b57fd250569fee648dff9e131192cd8bd2e957b

k8s-ci-robot avatar May 14 '24 13:05 k8s-ci-robot

/test pull-cluster-api-provider-vsphere-e2e-supervisor-main

chrischdi avatar May 14 '24 13:05 chrischdi

/test pull-cluster-api-provider-vsphere-e2e-supervisor-main

chrischdi avatar May 14 '24 13:05 chrischdi

I would guess linux vs mac base64 binary

sbueringer avatar May 14 '24 13:05 sbueringer

/test pull-cluster-api-provider-vsphere-e2e-supervisor-main

/test pull-cluster-api-provider-vsphere-e2e-govmomi-main

chrischdi avatar May 14 '24 14:05 chrischdi

failed to get component source YAML from URL: failed to get https://github.com/kubernetes-sigs/cluster-api/releases/download/v1.6.4/core-components.yaml: got status code 502

/retest

chrischdi avatar May 14 '24 15:05 chrischdi

/hold cancel

solved.

chrischdi avatar May 14 '24 16:05 chrischdi

Thx. Really appreciate the work on this!

/lgtm /approve

sbueringer avatar May 14 '24 17:05 sbueringer

LGTM label has been added.

Git tree hash: f1caa29b3e424fe6996efaebea83c001c453df3b

k8s-ci-robot avatar May 14 '24 17:05 k8s-ci-robot

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbueringer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar May 14 '24 17:05 k8s-ci-robot

Tests are failing due to the on-going capi release.

See:

  • https://github.com/kubernetes-sigs/cluster-api/issues/10450

chrischdi avatar May 14 '24 18:05 chrischdi

/retest

sbueringer avatar May 15 '24 08:05 sbueringer

/cherry-pick release-1.10

chrischdi avatar May 15 '24 11:05 chrischdi

/cherry-pick release-1.9

chrischdi avatar May 15 '24 11:05 chrischdi

/cherry-pick release-1.8

chrischdi avatar May 15 '24 11:05 chrischdi

/cherry-pick release-1.7

chrischdi avatar May 15 '24 11:05 chrischdi

@chrischdi: #2989 failed to apply on top of branch "release-1.9":

Applying: hack/e2e don't add binary files to artifacts and also censor base64 encoded values
Using index info to reconstruct a base tree...
M	hack/e2e.sh
Falling back to patching base and 3-way merge...
Auto-merging hack/e2e.sh
CONFLICT (content): Merge conflict in hack/e2e.sh
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 hack/e2e don't add binary files to artifacts and also censor base64 encoded values
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-1.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@chrischdi: #2989 failed to apply on top of branch "release-1.8":

Applying: hack/e2e don't add binary files to artifacts and also censor base64 encoded values
Using index info to reconstruct a base tree...
M	hack/e2e.sh
Falling back to patching base and 3-way merge...
Auto-merging hack/e2e.sh
CONFLICT (content): Merge conflict in hack/e2e.sh
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 hack/e2e don't add binary files to artifacts and also censor base64 encoded values
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-1.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@chrischdi: #2989 failed to apply on top of branch "release-1.7":

Applying: hack/e2e don't add binary files to artifacts and also censor base64 encoded values
Using index info to reconstruct a base tree...
M	hack/e2e.sh
Falling back to patching base and 3-way merge...
Auto-merging hack/e2e.sh
CONFLICT (content): Merge conflict in hack/e2e.sh
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 hack/e2e don't add binary files to artifacts and also censor base64 encoded values
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-1.7

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@chrischdi: new pull request created: #3005

In response to this:

/cherry-pick release-1.10

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.