Has anyone successfully run in AWS Lambda?

[DiMiTriFrog]

i cant run in lambda because i have a issue permissions error in '/home/user8353/

I change the user_dir, executabe paht and include I try to. put my chromedriver binary with this config path, but. nothing. Always is showing the error.

[diegoparrilla]

I made it work and I created this: https://github.com/diegoparrilla/headless-chrome-aws-lambda-layer

Changing from the vanilla Selenium it's almost straight forward. Hope it helps.

[DiMiTriFrog]

Hi @diegoparrilla thank you very much its so useful, but i need the same but, with undetected_chromedriver, the issue i have is for this library

[ottogutierrez]

@diegoparrilla I'm looking into doing this as well. Would you be able to give me some direction as to how to modify it to use "undetected-chromedriver"? thanks!

[Pedro-Dellazzari]

Has anyone managed to resolve this issue? I managed to run Selenium with Chromedriver installed in Lambda but I can't use Undetected-chromedriver yet

[arik103]

Any solutions to this? Thanks!

[kauannhenrique]

Same problem here, anyone solved?

[tanhaeirad]

Same here, Is there any update for this issue?

[DiMiTriFrog]

Waiting for solution...

[panterozo]

The full error. It seems to have something related to multiprocessing in AWS lambda

Traceback (most recent call last):
File "/app/main.py", line 9, in <module>
import undetected_chromedriver as uc
File "/usr/local/lib/python3.10/site-packages/undetected_chromedriver/__init__.py", line 43, in <module>
from .patcher import IS_POSIX
File "/usr/local/lib/python3.10/site-packages/undetected_chromedriver/patcher.py", line 25, in <module>
class Patcher(object):
File "/usr/local/lib/python3.10/site-packages/undetected_chromedriver/patcher.py", line 26, in Patcher
lock = Lock()
File "/usr/local/lib/python3.10/multiprocessing/context.py", line 68, in Lock
return Lock(ctx=self.get_context())
File "/usr/local/lib/python3.10/multiprocessing/synchronize.py", line 162, in __init__
SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
File "/usr/local/lib/python3.10/multiprocessing/synchronize.py", line 57, in __init__
sl = self._semlock = _multiprocessing.SemLock(
FileNotFoundError: [Errno 2] No such file or directory
Traceback (most recent call last):
File "/app/main.py", line 9, in <module>
import undetected_chromedriver as uc
File "/usr/local/lib/python3.10/site-packages/undetected_chromedriver/__init__.py", line 43, in <module>
from .patcher import IS_POSIX
File "/usr/local/lib/python3.10/site-packages/undetected_chromedriver/patcher.py", line 25, in <module>
class Patcher(object):
File "/usr/local/lib/python3.10/site-packages/undetected_chromedriver/patcher.py", line 26, in Patcher
lock = Lock()
File "/usr/local/lib/python3.10/multiprocessing/context.py", line 68, in Lock
return Lock(ctx=self.get_context())
File "/usr/local/lib/python3.10/multiprocessing/synchronize.py", line 162, in __init__
SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
File "/usr/local/lib/python3.10/multiprocessing/synchronize.py", line 57, in __init__
sl = self._semlock = _multiprocessing.SemLock(
FileNotFoundError: [Errno 2] No such file or directory

[HerreraCarlos81]

Getting the same error. Has anyone found a solution? can we disable multiprocessing on UC or enable it on AWS?

[filipopo]

Getting the same error. Has anyone found a solution? can we disable multiprocessing on UC or enable it on AWS?

Hey @HerreraCarlos81, fortunately I've done this recently It's not hard to make a patch to disable multiprocessing on the latest version but here is a makeshift guide: https://stackoverflow.com/questions/74290919/runnig-undetected-chromdriver-on-aws-lambda-message-e39098076c0be4f2-chromedr/74299965#74299965

[SweeneyBoy]

Getting the same error. Has anyone found a solution? can we disable multiprocessing on UC or enable it on AWS?

Hey @HerreraCarlos81, fortunately I've done this recently It's not hard to make a patch to disable multiprocessing on the latest version but here is a makeshift guide: https://stackoverflow.com/questions/74290919/runnig-undetected-chromdriver-on-aws-lambda-message-e39098076c0be4f2-chromedr/74299965#74299965

HI @filipopo I've been looking through the stackoverflow post you sent,

I can see your post mentioning downgrading the version of undetected driver and selenium, (as well as the copyong of the driver to /tmp) I was able to my aws lambda passed the "_multiprocessing.SemLock( " error mentioned above with this set up . But it then fails with "webdriverexception cant connect to chrome at 127.0.0.1:<PORT> from chrome: chrome not reachable" . Where you able to get this up and running based on the initial users set up? If so , can you share the configuration that you had for this.

THanks in advance

[marioalc19]

Getting the same error. Has anyone found a solution? can we disable multiprocessing on UC or enable it on AWS?

I'm on the same point, any other solution?

[filipopo]

Hey @SweeneyBoy Sorry for my delayed response, on which setup are you trying to base it on?

For me I based it on this repo https://github.com/umihico/docker-selenium-lambda

[SweeneyBoy]

HI @filipopo , I was using that docker set up as a base.

I've went back and started from the beginning, and I am able to get this to work now. It looks like I was missing some of the options that were suggested by you and esamhassan over on stackoverflow.

If anyone wants a working example then ping me and I'll provide my basic working set up

[filipopo]

@SweeneyBoy I felt bad for saying 2 weeks ago that I will push a fix for this so I did both that #1550 and made a (somewhat) minimal working example with instructions to set it up: https://github.com/filipopo/undetected-chromedriver-lambda

This allows you to use newer versions of undetected-chromedriver and selenium

The only reason it's "somewhat" minimal is because I'm not exactly sure which options are needed besides the headless mode of course, it doesn't work only with the headless mode for sure

[SweeneyBoy]

@SweeneyBoy I felt bad for saying 2 weeks ago that I will push a fix for this so I did both that #1550 and made a (somewhat) minimal working example with instructions to set it up: https://github.com/filipopo/undetected-chromedriver-lambda

This allows you to use newer versions of undetected-chromedriver and selenium

The only reason it's "somewhat" minimal is because I'm not exactly sure which options are needed besides the headless mode of course, it doesn't work only with the headless mode for sure

For me it looks like the options uc_options.add_argument('--single-process') uc_options.add_argument('--disable-dev-shm-usage') are key to getting it to run on lambda

[filipopo]

Thanks, you can write that down here: https://github.com/umihico/docker-selenium-lambda/issues/186 The dev there is saying that some more options are essential too, I'll try to test it out in the following days

[grantrosse]

Here's a minimal example (using a headed browser) based on filipopo's stackoverflow answer: https://github.com/grantrosse/headed-undetected-lambda-114