crawlee icon indicating copy to clipboard operation
crawlee copied to clipboard

Create generic function to perform login to page using Puppeteer

Open jancurn opened this issue 7 years ago • 3 comments

Most login pages just have username/email, password and submit button. We could write a function like Apify.utils.puppeteer.login() that would try to find these fields, fill them with provided values and then click submit or other button. It will not work all the time, but even 80% time is good enough. BTW browsers are also able to pre-fill login info, so maybe it's not that complicated after all.

jancurn avatar Nov 13 '18 15:11 jancurn

There are SO many people asking about this all the time. Once we have it in the SDK, we could even it add it as a special feature for our actor crawlers. I'm increasing the priority.

jancurn avatar Jan 11 '19 18:01 jancurn

Forms with inputs still seem to be the primary and most reliable implementation of logins relied on by browsers according to internet wisdom and source codes so the first iteration of this function should be able to cover a vast majority of logins using this approach and then we could try to reverse engineer the code from chromium for example to extend the functionality to the remaining edge cases.

sources: https://dxr.mozilla.org/firefox/source/toolkit/components/passwordmgr/src/nsLoginManager.js#626 https://github.com/scrapy/loginform/blob/master/loginform.py https://github.com/chromium/chromium/blob/master/components/password_manager/core/browser/form_parsing/form_parser.cc

cybairfly avatar Nov 04 '19 08:11 cybairfly

I think we are limited to using solely puppeteer methods instead of evaluating elements from within the browser since conversion between JShandles and ElementHandles doesn't seem to be possible and we need an ElementHandle to be able to .type on it in order to avoid having to change the input values directly or dealing with keycodes which would likely not work due to: https://stackoverflow.com/questions/13944835/how-to-simulate-typing-in-input-field-using-jquery

The closest thing I found resembling conversion JShandle -> ElementHandle was this: https://github.com/GoogleChrome/puppeteer/issues/537#issuecomment-334918553

Alternatively, we could use CDP for the typing as it's also a fallback method in .type Not sure this is ideal though, as the feature is tagged Experimental https://chromedevtools.github.io/devtools-protocol/tot/Input#method-insertText

cybairfly avatar Nov 05 '19 08:11 cybairfly