workload-discovery-on-aws icon indicating copy to clipboard operation
workload-discovery-on-aws copied to clipboard

Amazon Athena and AWS Glue Stack and GremlinStack Failure |

Open KatTraxler opened this issue 3 years ago • 8 comments

Both Stacks Fail with the same event. Gremlin Function Failure: Resource handler returned message: "Specified ReservedConcurrentExecutions for function decreases account's UnreservedConcurrentExecution below its minimum value of [10]. (Service: Lambda, Status Code: 400 AWSCURInitializer Failure: Resource handler returned message: "Specified ReservedConcurrentExecutions for function decreases account's UnreservedConcurrentExecution below its minimum value of [10]. (Service: Lambda, Status Code: 400, Request ID: 390a28cf-2d07-4dcb-abc3-8dc0d39a4ca2, Extended Request ID: null)"

I put in a quote request for concurrent executions on the Service Quota Page. Would take clear up this error?

Current Lambda Account Settings:

aws lambda get-account-settings

{
    "AccountLimit": {
        "TotalCodeSize": 80530636800,
        "CodeSizeUnzipped": 262144000,
        "CodeSizeZipped": 52428800,
        "ConcurrentExecutions": 10,
        "UnreservedConcurrentExecutions": 10
    },
    "AccountUsage": {
        "TotalCodeSize": 18965615,
        "FunctionCount": 8
    }
}

KatTraxler avatar Jan 17 '22 03:01 KatTraxler

Hi,

Yes, if you put in a service quota request to increase this it should fix the issue you are seeing.

Cheers

thebutler12 avatar Jan 18 '22 10:01 thebutler12

Thank you! Do you know what the minimum concurrent execution quota one needs to deploy this stack? -Kat

KatTraxler avatar Jan 18 '22 17:01 KatTraxler

Hey,

The GremlinStack Lambda needs 200 concurrent execution capacity. We haven't tested it with anything lower than that. The time taken for the Discovery Process to complete will increase if a lower value is used.

Cheers

thebutler12 avatar Jan 19 '22 13:01 thebutler12

Thanks.
After AWS support increased my quota, I am only seeing one failure now. Still from the AWS Perspective Gremlin Lambda Stack. GremlinFunction: Resource handler returned message: "Specified ReservedConcurrentExecutions for function decreases account's UnreservedConcurrentExecution below its minimum value of [100]. (Service: Lambda, Status Code: 400, Request ID: 8ebe7161-563e-4d10-9e33-efd5f8f3f371, Extended Request ID: null)" (RequestToken: 85e6d953-0d52-41d5-09a9-46e38b32b607, HandlerErrorCode: InvalidRequest)

Current Lambda Account Settings:

aws lambda get-account-settings

{
    "AccountLimit": {
        "TotalCodeSize": 80530636800,
        "CodeSizeUnzipped": 262144000,
        "CodeSizeZipped": 52428800,
        "ConcurrentExecutions": 200,
        "UnreservedConcurrentExecutions": 198
    },
    "AccountUsage": {
        "TotalCodeSize": 20938724,
        "FunctionCount": 9
    }
}

KatTraxler avatar Jan 20 '22 18:01 KatTraxler

That looks like the limit for the account is still to low. What did you increase your concurrency limit to?

thebutler12 avatar Jan 25 '22 16:01 thebutler12

I requested the quota be increased to 200

KatTraxler avatar Jan 25 '22 16:01 KatTraxler

You will need it to be higher than 200. This is because the GremlinFunction will take that 200, leaving 0 for the remaining functions.

thebutler12 avatar Jan 27 '22 11:01 thebutler12

I have the same error in DataDog forwarder. I've requested AWS Lambda "Concurrent executions" from 50 to 130. but I still get this error.

Resource handler returned message: "Specified ReservedConcurrentExecutions for function decreases account's UnreservedConcurrentExecution below its minimum value of [100]. (Service: Lambda, Status Code: 400, Request ID: 1d4340d5-f75c-41fe-b3f3-dba960a4506f)" (RequestToken: a73550e1-7e44-6014-24cf-bcfc6c478d89, HandlerErrorCode: InvalidRequest)

Yamkela-xaka avatar Jul 12 '22 08:07 Yamkela-xaka

Closing due to inactivity. Please feel free to reopen.

svozza avatar Dec 14 '22 11:12 svozza