Has anyone successfully run rod in AWS Lambda?

[DiMiTriFrog]

I would like to know if anyone has run the rod in Aws Lambda, because I configured a Lambda function with Python and are so troubles in the way for run correctly de function.

Anyone who has done it and can share their experience, configuration (e.g. chrome options (--single-process), that it needs to run well in Lambda or if anyone has tried running the headless-chromium binary with rod in Lambda.

Thanks!

[rod-robot]

Please add a valid **Rod Version:** v0.0.0 to your issue. Current version is v0.101.2 _{generated by check-issue}

[ysmood]

Need some time to investigate it.

[playniuniu]

AWS Lambda support custom docker images now

You may try to use custom docker images with chromium and go-rod.

The detail document is below:

https://docs.aws.amazon.com/lambda/latest/dg/lambda-images.html

[ysmood]

I tried to use the image, seems like AWS lambda has a lot of limitations for a docker container, such as you can't use the root user, and memory size is very limited. Our main target is to create an AWS lambda compatible demo Dockerfile.

[DiMiTriFrog]

I finally managed to get both a go binary and a docker running on lambda. Now I have a problem that I am trying to figure out but there is no information about it, both those two functions that I have discussed and a third one that I made with python + arsenic, have periods where they stop working and periods where they work just fine. In local those periods do not work (even in local docker), but there is something in Lambda that does not end up working well, I have paid to the support to see if they can help me but so far I have not gotten an answer.

I share with all you my configuration Docker and files.

Docker Go

hello-world.go

Dockerfile

headless-chromium In the same folder of the docker I have the headless-chromium I use get from: https://github.com/adieuadieu/serverless-chrome/releases

go.mod

go.sum

Direct Go binary

Now, for the direct Go Runtime I used a a Lambda Layer that contains the headless-chromium. For make the binary I use the follow command: GOARCH=amd64 GOOS=linux go build filename.go

main.go (I have configurated in Lambda main binary like the binary should been executed

Well, this functions works perfectly (include other functions do the same but with python + other libraries like arsenic), but periodically Lambda shoot errors:

An example of error:

The % of errors) When the lambda functions is in hight period of errors, locally the functions works normally.

Maybe the errors could be related to network saturation, maybe something specific needs to be configured in the VPC group. By solving this we could run go-rod functions in lambda without any problem.

If you need help in any configuration of external IP, VPC, private-subred and public, or other tell me.

[DiMiTriFrog]

The premium support has answer me (really good answer) and at this moment we think my function is not closing the connections when the function finish the run, the VPC is right configured and is working fine. Any way to close browser / proxy connection etc?

[pthieu]

@DiMiTriFrog were you able to make any progress on this issue?

[YoungiiJC]

Here's a working example I spun up, if anyone needs some help: https://github.com/YoungiiJC/go-rod-aws-lambda

[ysmood]

@YoungiiJC This is awesome, thank you!

[pthieu]

@YoungiiJC awesome! I looked through your code, looks like you're using a docker container to host the function and run it on Lambda, are you also using ECS to host the container image?

[YoungiiJC]

@pthieu yup. I'm deploying via the SAM cli, which automatically pushes the image to ECR.

Behind the scenes SAM glues everything together declared in the template.yaml file.

[pthieu]

@YoungiiJC solid.

In this case, because of ECR, there would be a cost incurred on the AWS account right?

[YoungiiJC]

@pthieu I suspect so

[ysmood]

@YoungiiJC I made a PR to slightly make it less resource consuming, can you help to check if it works as expect:

https://github.com/YoungiiJC/go-rod-aws-lambda/pull/1