community icon indicating copy to clipboard operation
community copied to clipboard

Panic when trying to adopt a lambda

Open mhr3 opened this issue 1 year ago • 3 comments

Describe the bug The lambda-controller crashed after applying a new AdoptedResource.

Logs

{"level":"info","ts":"2024-01-07T21:08:30.609Z","logger":"ackrt","msg":"desired resource state has changed","account":"***","role":"","region":"eu-west-1","kind":"Function","namespace":"cloud-dev","name":"ack-adopted-lambda","is_adopted":true,"generation":1,"diff":[{"Path":{"Parts":["Spec","Tags"]},"A":{"organization":"axiom","services.k8s.aws/controller-version":"lambda-1.3.5","services.k8s.aws/namespace":"cloud-dev","team":"eventflow"},"B":{"organization":"axiom","team":"eventflow"}}]}
{"level":"info","ts":"2024-01-07T21:08:30.610Z","msg":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","controller":"function","controllerGroup":"lambda.services.k8s.aws","controllerKind":"Function","Function":{"name":"ack-adopted-lambda","namespace":"cloud-dev"},"namespace":"cloud-dev","name":"ack-adopted-lambda","reconcileID":"536d2db5-fa89-4147-bbcf-7a06029bb1fb"}
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x18ecea3]

goroutine 792 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119 +0x1e5
panic({0x1b09440?, 0x31b4b20?})
    /usr/local/go/src/runtime/panic.go:914 +0x21f
github.com/aws-controllers-k8s/lambda-controller/pkg/resource/function.(*resourceManager).updateFunctionTags(0xc0008eb1e0, {0x22f3d70, 0xc000e2a300}, 0xc0007a24e8, 0xc0006963c8)
    /github.com/aws-controllers-k8s/lambda-controller/pkg/resource/function/hooks.go:333 +0x703
github.com/aws-controllers-k8s/lambda-controller/pkg/resource/function.(*resourceManager).customUpdateFunction(0xc000e2a330?, {0x22f3d70, 0xc000e2a300}, 0xc0006963c8, 0xc0009f3b00?, 0xc000c49740)
    /github.com/aws-controllers-k8s/lambda-controller/pkg/resource/function/hooks.go:77 +0x265
github.com/aws-controllers-k8s/lambda-controller/pkg/resource/function.(*resourceManager).sdkUpdate(...)
    /github.com/aws-controllers-k8s/lambda-controller/pkg/resource/function/sdk.go:951
github.com/aws-controllers-k8s/lambda-controller/pkg/resource/function.(*resourceManager).Update(0xc000e2a2d0?, {0x22f3d70?, 0xc000e2a300?}, {0x22fc3a8?, 0xc0006963c8?}, {0x22fc3a8?, 0xc0007a24e8}, 0x5?)
    /github.com/aws-controllers-k8s/lambda-controller/pkg/resource/function/manager.go:157 +0x78
github.com/aws-controllers-k8s/runtime/pkg/runtime.(*resourceReconciler).updateResource(0xc000948680, {0x22f3d70, 0xc000e2a300}, {0x22fc340, 0xc0008eb1e0}, {0x22fc3a8, 0xc0006963c8}, {0x22fc3a8, 0xc0007a24e8})
    /go/pkg/mod/github.com/aws-controllers-k8s/[email protected]/pkg/runtime/reconciler.go:536 +0x3d2
github.com/aws-controllers-k8s/runtime/pkg/runtime.(*resourceReconciler).Sync(0xc000948680, {0x22f3d70, 0xc000e2a300}, {0x22fc340, 0xc0008eb1e0}, {0x22fc3a8, 0xc000696398})
    /go/pkg/mod/github.com/aws-controllers-k8s/[email protected]/pkg/runtime/reconciler.go:279 +0x7de
github.com/aws-controllers-k8s/runtime/pkg/runtime.(*resourceReconciler).reconcile(0xc000948680, {0x22f3d70, 0xc000e2a300}, {0x22fc340, 0xc0008eb1e0}, {0x22fc3a8, 0xc000696398})
    /go/pkg/mod/github.com/aws-controllers-k8s/[email protected]/pkg/runtime/reconciler.go:215 +0x1cd
github.com/aws-controllers-k8s/runtime/pkg/runtime.(*resourceReconciler).Reconcile(0xc000948680, {0x22f3d70, 0xc000ddfbc0}, {{{0xc000630690?, 0x0?}, {0xc0009b44e0?, 0x4105a5?}}})
    /go/pkg/mod/github.com/aws-controllers-k8s/[email protected]/pkg/runtime/reconciler.go:186 +0x75b
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x22f3d70?, {0x22f3d70?, 0xc000ddfbc0?}, {{{0xc000630690?, 0x19f7340?}, {0xc0009b44e0?, 0xc00087d560?}}})
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:122 +0xb7
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000b30e60, {0x22f3da8, 0xc000600be0}, {0x1b9dda0?, 0xc000d4a500?})
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:323 +0x353
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000b30e60, {0x22f3da8, 0xc000600be0})
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274 +0x1c9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235 +0x79
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 553
    /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:231 +0x565

Environment

  • Kubernetes version: 1.26
  • Using EKS (yes/no), if so version? yes, 1.26
  • AWS service targeted (S3, RDS, etc.) lambda

This is the manifest I applied:

apiVersion: services.k8s.aws/v1alpha1
kind: AdoptedResource
metadata:
  name: adopted-lambda
spec:
  aws:
    nameOrID: lambda-test
  kubernetes:
    group: lambda.services.k8s.aws
    kind: Function
    metadata:
      name: ack-adopted-lambda
      namespace: cloud-dev

mhr3 avatar Jan 07 '24 21:01 mhr3

I should add that after the controller restarted, the function was adopted successfully.

mhr3 avatar Jan 07 '24 21:01 mhr3

Issues go stale after 180d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 60d of inactivity and eventually close. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle stale

ack-bot avatar Jul 08 '24 20:07 ack-bot

Stale issues rot after 60d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 60d of inactivity. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle rotten

ack-bot avatar Sep 06 '24 22:09 ack-bot

Rotten issues close after 60d of inactivity. Reopen the issue with /reopen. Provide feedback via https://github.com/aws-controllers-k8s/community. /close

ack-bot avatar Nov 05 '24 23:11 ack-bot

@ack-bot: Closing this issue.

In response to this:

Rotten issues close after 60d of inactivity. Reopen the issue with /reopen. Provide feedback via https://github.com/aws-controllers-k8s/community. /close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ack-prow[bot] avatar Nov 05 '24 23:11 ack-prow[bot]