urlwatch icon indicating copy to clipboard operation
urlwatch copied to clipboard

Proposals for "navigate": ability to set a custom user agent and to execute a javascript in the page context

Open maiux opened this issue 3 years ago • 2 comments

I like this project, if you agree I think this two features will be an improvement, since some websites knock down the request when they spot an "*/headless" user agent.

I hope it will be easy to set a custom user agent and evaluate javascript in the _get_content() function of browser module using page.setUserAgent and page.evaluate method exposed.

I think also it will be a great addition using playwright as an alternative browser interface, since playwright IMHO has a much smoother experience handling automation and has a native python API

maiux avatar Sep 08 '21 06:09 maiux

Both suggestions sound fine to me, patches gladly accepted.

thp avatar Nov 07 '21 07:11 thp

In #700 I added an optional key for changing the useragent (but did nothing about the javascript evaluation).

fversaci avatar Apr 07 '22 18:04 fversaci