micronaut-aws icon indicating copy to clipboard operation
micronaut-aws copied to clipboard

Memory leak in native image for AWS Lambda

Open msolanikova opened this issue 4 years ago • 4 comments

Max Memory Used keeps increasing when micronaut Serverless application is deployed to AWS Lambda as a native image.

Steps to Reproduce

  1. Follow instructions in official "Deploy a Micronaut function as a GraalVM Native Image to AWS Lambda" guide (https://guides.micronaut.io/mn-serverless-function-aws-lambda-graalvm/guide/index.html)
  2. Test the created AWS lambda function multiple times (step 3.4 in the guide)
  3. Check AWS CloudWatch logs and notice "Max Memory Used" increasing until OOM

Expected Behaviour

"Max Memory Used" should NOT increase

Actual Behaviour

"Max Memory Used" is increasing until OOM. Even after increasing -Xmx in bootstrap to 512m, memory usage is still increasing. However it is less rapid with more and more requests (e.g. after reaching 372MB it takes ~7 requests to get to 373MB) but it is still raising and doesn't ever go down.

Environment Information

  • Operating System: AWS Lambda (native image)
  • Micronaut Version: 2.0.1
  • JDK Version: java11

Example Application

https://github.com/micronaut-guides/mn-serverless-function-aws-lambda-graalvm

msolanikova avatar Nov 25 '20 14:11 msolanikova

Can you try adding -Dio.netty.allocator.numDirectArenas=0 -Dio.netty.noPreferDirect=true as arguments

graemerocher avatar Nov 25 '20 14:11 graemerocher

With these arguments in the bootstrap, the memory usage is much lower

  • without those arguments and using -Xmx512m, Max Memory Usage for multiple requests was: 148 MB, 182 MB, 217 MB, 251 MB, 285 MB, 320 MB, 354 MB, ...
  • with those arguments and using -Xmx512m, Max Memory Usage for multiple requests was: 99 MB, 101 MB, 103 MB, 105 MB, 107 MB, 109 MB, 111 MB, ...

So it's still increasing.

msolanikova avatar Nov 25 '20 15:11 msolanikova

Having the same issue. Getting memory error on 4th request every time

deepakthankachan avatar Apr 10 '21 17:04 deepakthankachan

removing the -Xmx128m option from bootstrap fixed the issue for me

https://github.com/micronaut-projects/micronaut-starter/issues/322

deepakthankachan avatar Apr 10 '21 17:04 deepakthankachan