requests-html icon indicating copy to clipboard operation
requests-html copied to clipboard

How to capture XHR requests?

Open raulblanko opened this issue 2 years ago • 5 comments

I'd like to capture XHR requests headers as well as JSON responses during page rendering

session = requests_html.HTMLSession()
r = session.get(url_sale_search)

r.html.render(keep_page=True, scrolldown=10, sleep=0.5)
page = r.html.page
browser = r.html.browser

Is there a way to do it? Also, is there a way to make browser visible?

raulblanko avatar May 18 '22 09:05 raulblanko

You could use Selenium for that, but it's perhaps overkill in most cases.

There is another way though. Whichever site you are on, you want to right click and go to Inspect From there, you will see a Network tab. Within this tab you will find XHR, just simply refresh the page and the requests will be there.

A youtuber by the name of John Watson Rooney goes in much better detail on this: https://www.youtube.com/watch?v=DqtlR0y0suo&t=638s Go to 0:50

El-Vim55 avatar Sep 08 '22 04:09 El-Vim55

are you kidding me? i was asking how to do it within a code

raulblanko avatar Sep 08 '22 08:09 raulblanko

Did you watch the entirety of the video? He goes through how to do it within a code... Although I did word the above comment wrong, implying that it was just done through the browser

El-Vim55 avatar Sep 08 '22 12:09 El-Vim55

unfortunately, you did not get what i asked for

never mind, forget about it, and please do not disturb me

raulblanko avatar Sep 08 '22 14:09 raulblanko