pipeline-aws-plugin icon indicating copy to clipboard operation
pipeline-aws-plugin copied to clipboard

cfnUpdate trows WaiterUnrecoverableException

Open hermesf opened this issue 6 years ago • 7 comments

Hello,

When trying to run cfnUpdate I get WaiterUnrecoverableException, but when creating the stack by the Amazon console it is created without problem

###Details: Version Pipeline: AWS Steps 1.27

I'm trying to execute: cfnUpdate(stack:"${stack}", url:"${urlTemplate}", params: 'roleName':"${roleName}",'bucket':"${bucket}",'pathS3':"${pathS3}",'handler':"${handler}"],timeoutInMinutes:10)

Where ${stack} es un el nombre de la pila ${urlTemplate} es el link a la plantilla guardada en S3

and throws in the jenkins log

com.amazonaws.waiters.WaiterUnrecoverableException: Resource never entered the desired state as it failed. at com.amazonaws.waiters.WaiterExecution.pollResource(WaiterExecution.java:78) at com.amazonaws.waiters.WaiterImpl.run(WaiterImpl.java:88) at com.amazonaws.waiters.WaiterImpl$1.call(WaiterImpl.java:110) at com.amazonaws.waiters.WaiterImpl$1.call(WaiterImpl.java:106) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused: java.util.concurrent.ExecutionException at org.apache.http.concurrent.BasicFuture.getResult(BasicFuture.java:71) at org.apache.http.concurrent.BasicFuture.get(BasicFuture.java:84) at de.taimos.pipeline.aws.cloudformation.EventPrinter.waitAndPrintEvents(EventPrinter.java:135) at de.taimos.pipeline.aws.cloudformation.EventPrinter.waitAndPrintStackEvents(EventPrinter.java:92) at de.taimos.pipeline.aws.cloudformation.CloudFormationStack.create(CloudFormationStack.java:119) at de.taimos.pipeline.aws.cloudformation.CFNUpdateStep$Execution.whenStackMissing(CFNUpdateStep.java:125) at de.taimos.pipeline.aws.cloudformation.AbstractCFNCreateStep$Execution$1.run(AbstractCFNCreateStep.java:137)

As a reference, my template is similar to: image

Maybe someone can help me with this or recommend me some adjustment?

regards

hermesf avatar Jul 18 '18 22:07 hermesf

does it take longer than 10 minutes? your timeout in minutes is 10

tylersouthwick avatar Jul 19 '18 17:07 tylersouthwick

Yes, I placed it in 10 minutes suspecting that the WaiterUnrecoverableException was due to that.

Creating the stack directly by   the AWS console, it takes less than 3 minutes

hermesf avatar Jul 19 '18 17:07 hermesf

Update.

My main class implements com.amazonaws.services.lambda.runtime.RequestHandler<Object, Object> and the main method is

@Override
public Object **handleRequest**(Request request, Context context) {...}

seeing the log
at com.amazonaws.waiters.WaiterImpl.run(WaiterImpl.java:88)

is this caused because my lambda code no implements the method run?

hermesf avatar Jul 27 '18 17:07 hermesf

I do not really understand what your source code has to so with the stack update failing?

Can you please add more of the Jenkins log.

hoegertn avatar Jul 30 '18 15:07 hoegertn

Of course,

In my Jenkins file :

  stage("PublishToProd") {
   steps {
   script{
   try{     
    echo "Parameter:"
    echo "stack:  ${stack}"
    echo "template:  ${urlTemplate}"
    echo "roleName: ${roleName}"
    echo "bucket: ${bucket}"
    echo "pathS3: ${pathS3}"
    echo "handler: ${handler}"
    echo "artifact: ${artifact}"
      
    def outputs = cfnUpdate(stack:"${stack}", url:"${urlTemplate}", params:['roleName':"${roleName}",'bucket':"${bucket}",'pathS3':"${pathS3}",'handler':"${handler}"],timeoutInMinutes:5)
    echo "outputs:    ${outputs}"    
        } catch (Throwable e) {
        echo "Exception: ${e.message}"
        throw e; 
    }
    }
   }

And in Console Output of jenkins, I got this:

[Pipeline] echo
stack:  MyStack
[Pipeline] echo
template:  https://s3.amazonaws.com/PathToTemplate.json
[Pipeline] echo
roleName: RoleName
[Pipeline] echo
bucket: EXAMPLE.us-west-1.wpt
[Pipeline] echo
pathS3: PathToFile.jar
[Pipeline] echo
handler: com.example.mainclass.Handler
[Pipeline] echo
artifact: target/file.jar
[Pipeline] cfnUpdate
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Stack: ActualizadorOdometro                                                                                                                                                                                                         |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[Pipeline] echo
Exception: com.amazonaws.waiters.WaiterUnrecoverableException: Resource never entered the desired state as it failed.
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] slackSend
run slackstepsend, step null:false, desc :false
Slack Send Pipeline step configured values from global config - baseUrl: true, teamDomain: true, token: true, channel: true, color: false
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
[BFA] Scanning build for known causes...
[BFA] Found failure cause(s):
[BFA] Some warnings
[BFA] Done. 0s
com.amazonaws.waiters.WaiterUnrecoverableException: Resource never entered the desired state as it failed.
at com.amazonaws.waiters.WaiterExecution.pollResource(WaiterExecution.java:78)
at com.amazonaws.waiters.WaiterImpl.run(WaiterImpl.java:88)
at com.amazonaws.waiters.WaiterImpl$1.call(WaiterImpl.java:110)
at com.amazonaws.waiters.WaiterImpl$1.call(WaiterImpl.java:106)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused: java.util.concurrent.ExecutionException
at org.apache.http.concurrent.BasicFuture.getResult(BasicFuture.java:71)
at org.apache.http.concurrent.BasicFuture.get(BasicFuture.java:84)
at de.taimos.pipeline.aws.cloudformation.EventPrinter.waitAndPrintEvents(EventPrinter.java:135)
at de.taimos.pipeline.aws.cloudformation.EventPrinter.waitAndPrintStackEvents(EventPrinter.java:92)
at de.taimos.pipeline.aws.cloudformation.CloudFormationStack.create(CloudFormationStack.java:119)
at de.taimos.pipeline.aws.cloudformation.CFNUpdateStep$Execution.whenStackMissing(CFNUpdateStep.java:125)
at de.taimos.pipeline.aws.cloudformation.AbstractCFNCreateStep$Execution$1.run(AbstractCFNCreateStep.java:137)

The template of Cloudformation is very similar to (I changed some parameters for hiding sensible info ): cfnTemplateExample.zip

Note: I tested my template in AWS console without problems

hermesf avatar Jul 30 '18 16:07 hermesf

Does the stack show up in the CloudFormation console? Which state does it have? Maybe you have to switch the filter to deleted stacks instead of active stacks.

hoegertn avatar Jul 30 '18 16:07 hoegertn

The Status of the stack is " UPDATE_COMPLETE" (in the active filter )

hermesf avatar Jul 30 '18 17:07 hermesf