snowplow
Make 'grunt packaging' use a different tmp dir
The output of my grunt commands:
vagrant@aws-lambda-example-project:/vagrant$ grunt init
Running "dynamo:default" (dynamo) task
{ TableDescription:
{ AttributeDefinitions: [ [Object], [Object], [Object] ],
CreationDateTime: Thu Jul 30 2015 20:47:16 GMT+0000 (UTC),
ItemCount: 0,
KeySchema: [ [Object], [Object] ],
LocalSecondaryIndexes: [ [Object] ],
ProvisionedThroughput:
{ NumberOfDecreasesToday: 0,
ReadCapacityUnits: 20,
WriteCapacityUnits: 20 },
TableName: 'my-table',
TableSizeBytes: 0,
TableStatus: 'CREATING' } }
Running "createRole:default" (createRole) task
{ ResponseMetadata: { RequestId: '29e6113c-36fc-11e5-a752-99498cbda2ea' },
StackId: 'arn:aws:cloudformation:us-east-1:901830632417:stack/kinesisDynamo/2a01aec0-36fc-11e5-b231-500150b34cb4' }
Running "kinesis:default" (kinesis) task
{}
Done, without errors.
vagrant@aws-lambda-example-project:/vagrant$ grunt role
Running "attachRole:default" (attachRole) task
Found
kinesisDynamo-LambdaExecRole-V1W0VC5RK5IR
Looking for ... kinesisDynamo-LambdaExecRole
Looking for ... kinesisDynamo-LambdaExecRole
{ ResponseMetadata: { RequestId: 'b934e8b8-3703-11e5-b79c-a50fb504c59a' } }
Running "packaging:default" (packaging) task
[email protected] dist/node_modules/aws-lambda-example-project
├── [email protected]
├── [email protected]
├── [email protected] ([email protected])
├── [email protected] ([email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected])
├── [email protected]
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected])
└── [email protected] ([email protected], [email protected], [email protected])
Created package at dist/aws-lambda-example-project_0-1-0_latest.zip
Done, without errors.
vagrant@aws-lambda-example-project:/vagrant$ grunt deploy
Running "deployLambda:default" (deployLambda) task
Found
arn:aws:iam::901830632417:role/kinesisDynamo-LambdaExecRole-V1W0VC5RK5IR
Looking for ... kinesisDynamo-LambdaExecRole
Looking for ... kinesisDynamo-LambdaExecRole
Polling for ARN
arn:aws:iam::901830632417:role/kinesisDynamo-LambdaExecRole-V1W0VC5RK5IR
Trying to create AWS Lambda Function...
Done, without errors.
vagrant@aws-lambda-example-project:/vagrant$ grunt connect
Running "associateStream:default" (associateStream) task
arn:aws:kinesis:us-east-1:901830632417:stream/my-stream
{ [InvalidParameterValueException: Function does not exist]
message: 'Function does not exist',
code: 'InvalidParameterValueException',
time: Thu Jul 30 2015 21:49:51 GMT+0000 (UTC),
statusCode: 400,
retryable: false,
retryDelay: 30 } 'InvalidParameterValueException: Function does not exist\n at Object.extractError (/vagrant/node_modules/aws-sdk/lib/protocol/json.js:43:27)\n at Request.extractError (/vagrant/node_modules/aws-sdk/lib/protocol/rest_json.js:37:8)\n at Request.callListeners (/vagrant/node_modules/aws-sdk/lib/sequential_executor.js:100:18)\n at Request.emit (/vagrant/node_modules/aws-sdk/lib/sequential_executor.js:77:10)\n at Request.emit (/vagrant/node_modules/aws-sdk/lib/request.js:604:14)\n at Request.transition (/vagrant/node_modules/aws-sdk/lib/request.js:21:12)\n at AcceptorStateMachine.runTo (/vagrant/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /vagrant/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request.<anonymous> (/vagrant/node_modules/aws-sdk/lib/request.js:22:9)\n at Request.<anonymous> (/vagrant/node_modules/aws-sdk/lib/request.js:606:12)'
Warning: Function does not exist Use --force to continue.
Aborted due to warnings.
Looks like the grunt task is running because it gets called ...
Running "associateStream:default" (associateStream) task
Lemme look into it...
The active call is the ...
lambda.createEventSourceMapping(params, function(err, data)
I will manually test this function...looks like the API just changed 2015-03-31 but I will have to confirm my hunch.
Hey Brian aka @BrianODell
When you log into the AWS Console Lambda tab does it have something like:
https://bigsnarf.files.wordpress.com/2015/07/screen-shot-2015-07-30-at-4-25-32-pm.png
But with a Lambda Function named ProcessKinesisRecordsDynamo ??? If yes, that would mean that the Lambda loaded correctly with the last grunt task.
If no, then the last task didnt upload the Lambda properly and it didn't get registered. Hopefully the name is the same too.
I do not have a lambda function named that. My only lambda function is one that I personally made.
@BrianODell,
There are several ways to get the function registered up in AWS Lambda Service. One way is manually upload the zip file. Another is to use the AWS CLI via this command:
aws lambda create-function
--region us-east-1
--function-name ProcessKinesisRecordsDynamo
--zip-file fileb://file-path/aws-lambda-example-project_0-1-0_latest.zip
--role arn:aws:iam::901830632417:role/kinesisDynamo-LambdaExecRole-V1W0VC5RK5IR
--handler ProcessKinesisRecords.handler
--runtime nodejs
--profile adminuser
--timeout 10
--memory-size 1024
And the third is the grunt script ... which is broken. Try uploading the function and either manually associate it to your Kinesis Stream or at the command line re-run grunt connect.
Let me know how you make out...Thanks for checking out the code.
Is the grunt script broken? I read this thread as Brian has changed the name of the Lambda function, so the hardcoded grunt scripts won't work for him...
Hi @alexanderdean
I had been playing with Lambda on my own before attempting to run this project.
I got it working! I noticed that I had no 'dist' folder, and managed to track it down to a bug in the 'packaging' task.
See my pull request: https://github.com/snowplow/aws-lambda-nodejs-example-project/pull/18
Thanks for all the responses. The project looks really great!
Thanks @BrianODell! I'll close your PR and just reference the fix in this ticket so that you don't have to go through the whole CLA rigmarole.
For fix: https://github.com/snowplow/aws-lambda-nodejs-example-project/pull/18/files
Thanks again!
It's my pleasure. Thanks for making this project. I intend to borrow lots of the grunt and ansible stuff from you. I love the setup of the project.
I have some ideas I'm going to work on to tweak this project. Might there be a good place to chat about this project besides the comments of this issue? Thanks again :)
Hey @BrianODell - sure thing, you can open a new thread in https://groups.google.com/forum/#!forum/snowplow-user
Look forward to your thoughts!
I was able to recreate the issue. I was able to verify this issue and was able to redirect the the target directory to _dist and pick up the _dist folder. I'm gonna try @BrianODell solution as it is more elegant.