NoDriver: how to get response body

[RobertAzovski]

I am struggling to be able extract body from a response using NoDriver. I didn't find docs or examples.

I found a method called get_response_body(requestId) but it returns None for all requests

[ven0ms99]

await page.get_content()

Here is how to get the title of page:

....
from selectolax.parser import HTMLParser
....

html = await page.get_content()
tree = HTMLParser(html)
title = tree.head.css("title")[0].html

[RobertAzovski]

@Hillcow Bruh, seems like you have not idea what I am talking about

[ultrafunkamsterdam]

Well, i know what you are talking about.

This is the wrong repositor for nodriver. It is not even released yet.

I didn't find docs or examples.

Apparently /example/network_monitor.py (https://github.com/ultrafunkamsterdam/nodriver/blob/main/example/network_monitor.py) is not clear enough.

from nodriver import *
async def main():
    browser = await start()
    tab = await browser.get('data:,')   # bogus page to get a tab object. could also use browser.main_tab
    tab.add_handler(cdp.network.ResponseReceived, myhandler)
    await tab.get('https://www.google.com')
    
async def myhandler(event: cdp.network.ResponseReceived):
    print(event.response)

# could also use regular callback like
#
#  def myhandler(event:  cdp.network.ResponseReceived):
#        print(event.response)


loop().run_until_complete(main())

[boludoz]

Well, i know what you are talking about.

This is the wrong repositor for nodriver. It is not even released yet.

I didn't find docs or examples.

Apparently /example/network_monitor.py (https://github.com/ultrafunkamsterdam/nodriver/blob/main/example/network_monitor.py) is not clear enough.

from nodriver import *
async def main():
    browser = await start()
    tab = await browser.get('data:,')   # bogus page to get a tab object. could also use browser.main_tab
    tab.add_handler(cdp.network.ResponseReceived, myhandler)
    await tab.get('https://www.google.com')
    
async def myhandler(event: cdp.network.ResponseReceived):
    print(event.response)

# could also use regular callback like
#
#  def myhandler(event:  cdp.network.ResponseReceived):
#        print(event.response)


loop().run_until_complete(main())

Can you help me move to a certain frame and click on an element?

[ultrafunkamsterdam]

Just search for the element. It searches in frames too

[RzNmKX]

I may be just dense today but I have tried a number of ways to use get_response_body() both on cdp.network and cdp.fetch to get the actual body of the response without luck. sure, it is no problem to get the entire html of the page. but suppose some data comes through in a XHR that is much easier to parse in it's native json than rendered as html. is this possible? the returned objects are of type 'generator'

this is almost entirely from the network_monitor.py example:

from nodriver import start, cdp, loop

async def main():
    browser = await start()
    tab = browser.main_tab
    # tab.add_handler(cdp.network.RequestWillBeSent, send_handler)

    tab.add_handler(cdp.network.ResponseReceived, receive_handler)
    tab = await browser.get("https://example.com")
    tab.wait()
    await tab.wait(20)

async def receive_handler(event: cdp.network.ResponseReceived):
    print("---------------------------------------")
    print(event.response)

    response_body = cdp.fetch.get_response_body(event.request_id)

    print(type(response_body))
    # <class 'generator'>

regarding the nodriver repo: An owner of this repository has limited the ability to open an issue to users that are collaborators on this repository.

[jwwq]

is this possible? the returned objects are of type 'generator'

you should pass this generator to send function (see pyCDP docs about commands and many examples in nodriver source code)

cdp_cmd = cdp.network.get_response_body(event.request_id)
res = await browser.main_tab.send(cdp_cmd)

but in this case it won't help ))) (see my issue https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/1832). Please let me know if you find a way to fix it