argo-cd icon indicating copy to clipboard operation
argo-cd copied to clipboard

fix: Bump Kubernetes 1.29.2

Open sivchari opened this issue 4 months ago • 25 comments

close #17490

closes https://github.com/argoproj/argo-cd/issues/16473

Checklist:

  • [ ] Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • [ ] The title of the PR states what changed and the related issues number (used for the release note).
  • [ ] The title of the PR conforms to the Toolchain Guide
  • [ ] I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • [ ] I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • [ ] Does this PR require documentation updates?
  • [ ] I've updated documentation as required by this PR.
  • [ ] I have signed off all my commits as required by DCO
  • [ ] I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • [ ] My build is green (troubleshooting builds).
  • [ ] My new feature complies with the feature status guidelines.
  • [ ] I have added a brief description of why this PR is necessary and/or what this PR solves.
  • [ ] Optional. My organization is added to USERS.md.
  • [ ] Optional. For bug fixes, I've indicated what older releases this fix should be cherry-picked into (this may or may not happen depending on risk/complexity).

sivchari avatar Mar 12 '24 23:03 sivchari

Both this commit and one more commit is to fix some errors by lint on CI, due to updating libraries.

sivchari avatar Mar 13 '24 01:03 sivchari

Hello @sivchari,

There seems to be some codegen issues:

Then run make codegen again - hopefully that should get you past that hurdle at least

blakepettersson avatar Apr 08 '24 21:04 blakepettersson

Hi, @blakepettersson

CI has still been failed and the log is below.

+ INT_APIS_PKG=
++ dirname /tmp/generate-groups.sh
+ exec /tmp/generate-internal-groups.sh deepcopy,client,informer,lister github.com/argoproj/argo-cd/v2/pkg/client '' github.com/argoproj/argo-cd/v2/pkg/apis application:v1alpha1 --go-header-file /home/runner/go/src/github.com/argoproj/argo-cd/hack/custom-boilerplate.go.txt --output-base /home/runner/go/src
/tmp/generate-groups.sh: line 66: /tmp/generate-internal-groups.sh: No such file or directory
make: *** [Makefile:219: clientgen] Error 127

What should I do ? Thanks.

sivchari avatar Apr 10 '24 02:04 sivchari

Hello @sivchari you need to do the two steps which I mentioned in my previous comment, namely

*Can you modify hack/update-codegen.sh to replace generate-groups.sh with kube_codegen.sh * In hack/install-codegen-go-tools.sh can you remove this line (line 45)?

blakepettersson avatar Apr 10 '24 08:04 blakepettersson

Hello @blakepettersson Should I contains these changes which you mentioned into commit ? I had already done two steps that you told me.

sivchari avatar Apr 12 '24 01:04 sivchari

Can I extend the timeout ?

sivchari avatar Apr 15 '24 08:04 sivchari

@sivchari the timeout is already exceeding ~2x of what the e2e tests normally take - there is a panic in e2e.TestAppSyncWrongVersion, either in the test itself or in the underlying code which is being tested which needs to be addressed

blakepettersson avatar Apr 15 '24 09:04 blakepettersson

the timeout is already exceeding ~2x of what the e2e tests normally take - there is a panic in e2e.TestAppSyncWrongVersion, either in the test itself or in the underlying code which is being tested which needs to be addressed

@blakepettersson I got it. I'd investigate, thanks.

sivchari avatar Apr 15 '24 09:04 sivchari

time="2024-05-01T16:08:56Z" level=info msg="../../dist/argocd app sync test-sync-with-force-replace --timeout 20 --prune --plaintext --server 127.0.0.1:8088 --auth-token *** --insecure" dir= execID=801d3
time="2024-05-01T16:08:56Z" level=debug msg="TIMESTAMP                  GROUP        KIND   NAMESPACE                                                      NAME    STATUS    HEALTH        HOOK  MESSAGE\n2024-05-01T16:08:56+00:00            Service  argocd-e2e--test-sync-with-force-replace-ngjnw          guestbook-ui    Synced   Healthy              \n2024-05-01T16:08:56+00:00   apps  Deployment  argocd-e2e--test-sync-with-force-replace-ngjnw          guestbook-ui  OutOfSync  Healthy              \n\nName:               argocd-e2e/test-sync-with-force-replace\nProject:            default\nServer:             [https://kubernetes.default.svc\nNamespace:](https://kubernetes.default.svc/nNamespace:)          argocd-e2e--test-sync-with-force-replace-ngjnw\nURL:                [http://127.0.0.1:8088/applications/test-sync-with-force-replace\nSource:\n-](http://127.0.0.1:8088/applications/test-sync-with-force-replace/nSource:/n-) Repo:             file:///tmp/argo-e2e/testdata.git\n  Target:           \n  Path:             guestbook\nSyncWindow:         Sync Allowed\nSync Policy:        Manual\nSync Status:        OutOfSync from  (423086c)\nHealth Status:      Healthy\n\nOperation:          Sync\nSync Revision:      423086c63f5cc4c14c68000bb1beb44dac4a76c2\nPhase:              Failed\nStart:              2024-05-01 16:08:56 +0000 UTC\nFinished:           2024-05-01 16:08:56 +0000 UTC\nDuration:           0s\nMessage:            one or more objects failed to apply, reason: error when replacing \"/dev/shm/180188768\": Deployment.apps \"guestbook-ui\" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{\"app\":\"guestbook-ui\", \"env\":\"e2e\"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable\n\nGROUP  KIND        NAMESPACE                                       NAME          STATUS     HEALTH   HOOK  MESSAGE\n       Service     argocd-e2e--test-sync-with-force-replace-ngjnw  guestbook-ui  Synced     Healthy        service/guestbook-ui unchanged\napps   Deployment  argocd-e2e--test-sync-with-force-replace-ngjnw  guestbook-ui  OutOfSync  Healthy        error when replacing \"/dev/shm/180188768\": Deployment.apps \"guestbook-ui\" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{\"app\":\"guestbook-ui\", \"env\":\"e2e\"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable\n" duration=299.288599ms execID=801d3
time="2024-05-01T16:08:56Z" level=error msg="`../../dist/argocd app sync test-sync-with-force-replace --timeout 20 --prune --plaintext --server 127.0.0.1:8088 --auth-token *** --insecure` failed exit status 1: time=\"2024-05-01T16:08:56Z\" level=fatal msg=\"Operation has completed with phase: Failed\"" execID=801d3

Odd... has the behavior of force-replace changed? Or maybe there's an issue that's broken force-replace in our code?

crenshaw-dev avatar May 01 '24 18:05 crenshaw-dev

error when replacing "/dev/shm/180188768": Deployment.apps "guestbook-ui" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"guestbook-ui", "env":"e2e"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable\

If these tests will be passed, but same log will be displayed, we can assume this bug is another issue. @crenshaw-dev What do you think ?

sivchari avatar May 02 '24 02:05 sivchari

@crenshaw-dev

https://github.com/argoproj/argo-cd/issues/12439

This error seems to be happened since the above.

sivchari avatar May 02 '24 07:05 sivchari

I guess this error depends on version of kubectl, because latest commit of ArgoCD works properly..

sivchari avatar May 02 '24 07:05 sivchari

OK, I solved it. The dependency which replaces gitops-engine didn't include this commit. Then, ArgoCD couldn't replace forcibly, probably because of kubectl is upgraded. So I made ArgoCD use gitops-engine including latest commit. It worked in local environmentental. Thus checks might have been completed later.

sivchari avatar May 02 '24 10:05 sivchari

Only what I should resolve might be in clusterauth.go. So I'd fix later 👍

sivchari avatar May 02 '24 10:05 sivchari

@backjo @crenshaw-dev All test is passed 👍 I've used my pull-request which is submitet to gitops-engine yet. So, when it is merged, this pull-request will be ready to merge. Thanks.

sivchari avatar May 02 '24 11:05 sivchari

@sivchari hmm, perhaps openapigen should be omitted after all in make codegen, but I don't think so 🤔

blakepettersson avatar May 02 '24 12:05 blakepettersson

@blakepettersson I don't have better way to solve this issue on my laptop, but I agree with your opinion. What should I do ??

sivchari avatar May 02 '24 12:05 sivchari

@blakepettersson If you feel free to take over my commit, could you do the part of codegen ? Presumably, it's problem with my environmental (of course, we should deal with this issue in the future).

sivchari avatar May 02 '24 12:05 sivchari

Some tests are failed due to rate limit too.

sivchari avatar May 03 '24 05:05 sivchari

Looks possibly related:

    deployment_test.go:322: 
        	Error Trace:	/home/runner/work/argo-cd/argo-cd/test/e2e/deployment_test.go:322
        	            				/home/runner/work/argo-cd/argo-cd/test/e2e/deployment_test.go:173
        	Error:      	Expected nil, but got: &fmt.wrapError{msg:"failed to create secret for serviceaccount \"\": an empty namespace may not be set during creation", err:(*errors.errorString)(0xc00045f410)}

crenshaw-dev avatar May 06 '24 17:05 crenshaw-dev

@crenshaw-dev Okay, I solved it. This problem was happened due to that we don't exit the loop if service account doesn't have secrets. The new function wait until returning value is true or non-nil error. Since we'll create new secret when service account doesn't have secret, I modify the logic that exit the loop even though service account doesn't have secret.

sivchari avatar May 07 '24 05:05 sivchari

@crenshaw-dev @blakepettersson All test are passed 😄

sivchari avatar May 07 '24 07:05 sivchari

Great work @sivchari!

blakepettersson avatar May 07 '24 09:05 blakepettersson

@crenshaw-dev I fixed and replied these nits.

sivchari avatar May 07 '24 16:05 sivchari

After it is merged, we can be ready to merge this PR.

sivchari avatar May 07 '24 17:05 sivchari

Merged!

crenshaw-dev avatar May 07 '24 20:05 crenshaw-dev

@crenshaw-dev I placed it.

sivchari avatar May 08 '24 01:05 sivchari