requests-html
requests-html copied to clipboard
How to capture XHR requests?
I'd like to capture XHR requests headers as well as JSON responses during page rendering
session = requests_html.HTMLSession()
r = session.get(url_sale_search)
r.html.render(keep_page=True, scrolldown=10, sleep=0.5)
page = r.html.page
browser = r.html.browser
Is there a way to do it? Also, is there a way to make browser visible?
You could use Selenium for that, but it's perhaps overkill in most cases.
There is another way though. Whichever site you are on, you want to right click and go to Inspect From there, you will see a Network tab. Within this tab you will find XHR, just simply refresh the page and the requests will be there.
A youtuber by the name of John Watson Rooney goes in much better detail on this: https://www.youtube.com/watch?v=DqtlR0y0suo&t=638s Go to 0:50
are you kidding me? i was asking how to do it within a code
Did you watch the entirety of the video? He goes through how to do it within a code... Although I did word the above comment wrong, implying that it was just done through the browser
unfortunately, you did not get what i asked for
never mind, forget about it, and please do not disturb me