Configure DynamoDB stream locally

[tgfischer]

trafficstars

I'm trying to trigger a lambda to update an Elastic Search instance whenever changes are made to DynamoDB. I have the stream set up in my serverless.yaml file, and when I deploy it, the lambda is triggered. However, when I run this with serverless-offline, the function is not triggered.

Here is my serverless.yaml

service: my-service

plugins:
  - serverless-dynamodb-local
  - serverless-offline

custom:
  tableName: "dynamodb-${self:provider.stage}"
  dynamodb:
    start:
      migrate: true

provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  region: ca-central-1
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:Query
        - dynamodb:Scan
        - dynamodb:GetItem
        - dynamodb:PutItem
        - dynamodb:UpdateItem
        - dynamodb:DeleteItem
        - dynamodb:DescribeStream
        - dynamodb:GetRecords
        - dynamodb:GetShardIterator
        - dynamodb:ListStreams
      Resource:
        - { "Fn::GetAtt": ["DynamoDbTable", "Arn" ] }

functions:
  # Request that should trigger function
  add:
    handler: handler.add
    events:
      - http:
          path: api/add
          method: POST
  # Function that should be triggered
  updateIndices:
    handler: handler.updateIndices
    events:
      - stream:
          type: dynamodb
          arn:
            Fn::GetAtt:
              - DynamoDbTable
              - StreamArn
          startingPosition: LATEST
          batchSize: 1
          enabled: true

resources:
  Resources:
    DynamoDbTable:
      Type: "AWS::DynamoDB::Table"
      DeletionPolicy: Delete
      Properties:
        TableName: ${self:custom.tableName}
        AttributeDefinitions:
          - AttributeName: id
            AttributeType: S
        KeySchema:
          - AttributeName: id
            KeyType: HASH
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        StreamSpecification:
          StreamViewType: NEW_AND_OLD_IMAGES
    ElasticSearchInstance:
      Type: "AWS::Elasticsearch::Domain"
      Properties:
        EBSOptions:
          EBSEnabled: true
          VolumeType: gp2
          VolumeSize: 10
        ElasticsearchClusterConfig:
          InstanceType: t2.small.elasticsearch
          InstanceCount: 1
          DedicatedMasterEnabled: false
          ZoneAwarenessEnabled: false
        ElasticsearchVersion: "6.0"

Is what I'm trying to do possible with serverless-dynamodb-local? Or is there more configuration that needs to be added to fire this function locally?

Thanks for the help!

[rehrumesh]

I had a look at the source and noticed that the streams are enabled even in the DDb local.

https://github.com/99xt/serverless-dynamodb-local/blob/265d3417b9f81665a1c9a3d3aa14ca969295335b/index.js#L241

Might need some time to inspect this.

[dankelleher]

Hi @rehrumesh, any luck on this? I am having exactly the same problem.

[matttowerssonos]

+1 here. @rehrumesh Were you ever able to find out more?

[dehian]

+1 i can't make it work either. don't know what i'm missing here

[mbudm]

Yeah I was hoping this was available too for streams as I would prefer to test out a new feature locally first. I'm taking a look into it now but my initial impression is that there's no obvious sign that serverless-dynamodb-local is passing any config to dynamodb-local about what lambdas should handle stream events.

Thinking about this now - that would really be a job of serverless-offline as api-gateway would do this in AWS. I'll do a bit more digging but I'm guessing the functionality is just not there and this is just something I'll have to dev against my stage environment in AWS

[mbudm]

Yeah I think the functionality isnt there without a plugin update either on this one or serverless-offline - this thread suggests serverless-offline doesnt support events. You could go the SNS workaround but I'm not keen on adding another resource just to get offline functionality ;-)

https://github.com/dherault/serverless-offline/issues/235

[mydeen1989]

Hi, I am also facing the same issue. I have tried the below plugins

   1. serverless-offline-dynamodb-streams
2. serverless-plugin-offline-dynamodb-stream

I am able see the data persisted in dynamodb local but the streams is not triggered. I am using serverless 3.0 framework.