pipeline-aws-plugin icon indicating copy to clipboard operation
pipeline-aws-plugin copied to clipboard

invokeLambda method causes Lambda to be executed four times

Open JPGrieb opened this issue 5 years ago • 5 comments

Description

I’m using the "invokeLambda" as part of my Jenkins pipeline to kick off a Lambda function that is taking approximately 50 to 70 seconds to complete.

The lambda function completes successfully each time.

The Lambda function is consistently getting executed four times instead of just once even though I am calling the “invokeLambda” method only once.

Each execution has a different execution id so it does not appear to be AWS retrying the function.

Steps to Reproduce

withAWS(region: "eu-west-1", roleAccount: '906291453955', role: 'a205470-JenkinsCrossAccount') {

     invokeLambda(functionName: 'a205470-wcol-update-monitors',
    payload: [ "environment" : "ppe02", "bucket" : "preprod-ppe02", "method" : "Create" ]))

}

Expected behavior: Lamba would be invoked once

Actual behavior: Lambda was invoked 4 times

Environment

Jenkins-Version: Jenkins ver. 2.190.2

Java-Version: ????

Plugin-Version: 1.36 Master/Slave Setup: No

JPGrieb avatar Nov 19 '19 19:11 JPGrieb

I have a difficulty similar to the @JPGrieb report

When invoking a Lambda function using the invokeLambda method, it's not waiting for the execution to return. In my case I'm doing this:

Jenkinspipeline:

stage('AWS Lambda'){
	steps {
		invokeLambda([awsRegion: 'us-east-1',
			functionName: 'function-lambda', 
			payload: "{ \"Suspend\": \"true\"}] }", 
			synchronous: true, 
			useInstanceCredentials: true])
	}
}

Lambda function code:

exports.handler = async (event, context) => {
	return new Promise(
        function(resolve, reject) {
			execMethod({ ... }, function(err, data) {
				if(err){
					reject(Error(err));
				}else{
					resolve("Done!");
				}
			})
		}
	)
}

Jenkins version: 1.24.3

Plugin version: 1.43

rafaeldalsenter avatar Dec 23 '20 19:12 rafaeldalsenter

@rafaeldalsenter Were you able to address the issue and wait for execution of lambda function to complete ? If yes, how did you achieve this ?

aditya-kapre-shrewsbury avatar Jun 09 '21 13:06 aditya-kapre-shrewsbury

Hi @adityakapre,

To solve this, I installed AWS CLI and used sh to invoke the lambda function:

aws lambda invoke --function-name function-lambda --payload \"{ \"Suspend\": \"true\"}] }\" --invocation-type RequestResponse response.json

rafaeldalsenter avatar Jun 09 '21 13:06 rafaeldalsenter

The reason why you see this behavior is cli\sdk retries: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-retries.html The issue can be reproduced in AWS CLI as well. If you have a long-running lambda (that runs more than 60 seconds), after 60 seconds first call will be treated as time-outed and will be repeated AWS_MAX_ATTEMPTS -1 times. To fix it for CLI, you need to run invoke command as follows: AWS_MAX_ATTEMPTS=1 aws lambda invoke --cli-read-timeout 300 ... This call will invoke lambda exactly one time and will error with a timeout after 5 minutes. Same configuration for retries can be done for Java SDK: clientConfiguration.setMaxErrorRetry(0) Unfortunately, I don't have time to raise a PR, but hope this comment helps.

tony-bondarenko avatar Aug 30 '21 19:08 tony-bondarenko

is this problem solved yet or i need to use a sh script to invoke lambda ?

MedMokhtarAmmar avatar Jun 20 '23 17:06 MedMokhtarAmmar