lift icon indicating copy to clipboard operation
lift copied to clipboard

Queue construct not working as documented

Open ledburyb opened this issue 3 years ago • 4 comments

Trying to adapt the documentation example so I can manage queues on dev differently but still use lift to create the production queues:

constructs:
  update-segment-memberships-from-contact-changes:
    type: queue

custom:
  update-segment-memberships-sqs-queue-arn:
    dev: ${env:UPDATE_SEGMENT_MEMBERSHIPS_FROM_CONTACT_CHANGES_QUEUE_ARN}
    staging: ${construct:update-segment-memberships-from-contact-changes.queueArn}
    demo: ${construct:update-segment-memberships-from-contact-changes.queueArn}
    production: ${construct:update-segment-memberships-from-contact-changes.queueArn}

functions:
  update_segment_memberships_from_contact_changes:
    events:
      - sqs:
          arn: ${self:custom.update-segment-memberships-sqs-queue-arn.${sls:stage}}
          batchSize: 10
          maximumBatchingWindow: 60

but getting:

Invalid configuration in 'constructs.update-segment-memberships-from-contact-changes': no 'worker' defined. Queue constructs require a 'worker' function to be defined.```

ledburyb avatar Oct 07 '21 14:10 ledburyb

This is the intended behaviour at the moment.

You need to define a worker key that configures the Lambda function that will handle SQS messages.

Something like this

constructs:
    my-queue:
        type: queue
        worker:
            handler: src/worker.handler

Where in the docs did you found it was not clear this was supposed to be used that way ?

That being said, I understand what you are trying to do but I'm not sure how that could work with the current implementation :thinking:

Could you give a bit more details about the use case ? Is this to run locally ?

t-richard avatar Oct 07 '21 15:10 t-richard

This was from the example here: https://github.com/getlift/lift/blob/master/docs/queue.md#variables I'm just trying to get something working on my dev machine where I can push to a queue and have the handler run but so far I haven't had much success with either localstack or trying to inject real SQS queues.

ledburyb avatar Oct 07 '21 15:10 ledburyb

I understood the same thing @ledburyb did from the docs. The usage would be that serverless will only publish it to the queue, and a different service will handle the messages from the queue. Was this ever addressed?

amit-toren avatar Feb 02 '23 09:02 amit-toren

@amit-toren no, Lift requires the worker (or "queue handler") to be defined in the same service.

mnapoli avatar Feb 02 '23 12:02 mnapoli