[🐛 Bug]: Can't get Fetch.GetResponseBody

[johneakin]

What happened?

Have tried with both 'Network' and 'Fetch' Am able to get the basic response but can't get the response body. I understand the request has to be stopped at the response stage , but even with this I can never get a response body returned.

Always get error: {"A command response was not received: Fetch.getResponseBody"}

How can we reproduce the issue?

domains = session.GetVersionSpecificDomains<DevToolsSessionDomains>();
            var enableFetchCommandSettings = new OpenQA.Selenium.DevTools.V96.Fetch.EnableCommandSettings();

            var requestPattern = new OpenQA.Selenium.DevTools.V96.Fetch.RequestPattern();
            requestPattern.RequestStage = OpenQA.Selenium.DevTools.V96.Fetch.RequestStage.Response;
            //requestPattern.ResourceType = ResourceType.XHR;
            enableFetchCommandSettings.Patterns = new OpenQA.Selenium.DevTools.V96.Fetch.RequestPattern[] { requestPattern };
            domains.Fetch.Enable(enableFetchCommandSettings);


            domains.Fetch.RequestPaused += RequestPaused;
            driver.Navigate().GoToUrl("https://www.google.com");




static void RequestPaused(object sender, OpenQA.Selenium.DevTools.V96.Fetch.RequestPausedEventArgs e)
        {
            

                try
                {
                var getResponseBody = domains.Fetch.GetResponseBody(new OpenQA.Selenium.DevTools.V96.Fetch.GetResponseBodyCommandSettings()
                {
                    RequestId = e.RequestId
                });

                //Note: RequestID always seems to be valued: '"interception-job-1.0"' 

                var body = getResponseBody.Result.Body;
                }
                catch (Exception ex)
                {
                    Console.WriteLine(ex);
                }

        }

Relevant log output

{"A command response was not received: Fetch.getResponseBody"}

Operating System

Windows 10

Selenium version

41

What are the browser(s) and version(s) where you see this issue?

Chrome 99

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver 99.0.4844.5100

Are you using Selenium Grid?

No response

[github-actions[bot]]

@johneakin, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[JamesWilliamsNeve]

I'm having the same issue here, regardless of how I try, I can never get the response body.

I checked in Chrome using a extension and "Network.getResponseBody" works in browser, but not via selenium

    public static void Main(string[] args)
    {
        ChromeDriver driver;

        var chromeOptions = new ChromeOptions();
        //chromeOptions.AddArgument("--incognito");
        chromeOptions.AddArgument("window-size=1280,800");


        driver = new ChromeDriver(chromeOptions);

        IDevTools devTools = driver as IDevTools;
        DevToolsSession session = devTools.GetDevToolsSession();
        session.SendCommand<EnableCommandSettings>(new EnableCommandSettings());
        devToolsSession = session.GetVersionSpecificDomains<DevToolsSessionDomains>();


        void RequestToBeSent(object sender, RequestWillBeSentEventArgs e)
        {
            Console.WriteLine("REQ: " + e.RequestId);
        }

        void RequestIntercepted(object sender, LoadingFinishedEventArgs e)
        {

            Console.WriteLine("RES: " + e.RequestId);

            try
            {
                Dictionary<string, object> cdpParameters = new Dictionary<string, object> { { "requestId", e.RequestId } };
                Dictionary<string, object> res = (Dictionary<string, object>)((ChromeDriver)driver).ExecuteCdpCommand("Network.getResponseBody", cdpParameters);


                if (res.TryGetValue("body", out object? bodyObj))
                {
                    string body = (string)bodyObj;
                    Console.WriteLine(body);
                }
            }
            catch (Exception ex)
            {
                // always hit
                Console.WriteLine(ex.Message + " " + e.RequestId);
            }


        }
        devToolsSession.Network.RequestWillBeSent += RequestToBeSent;

        devToolsSession.Network.LoadingFinished += RequestIntercepted;


        driver.Navigate().GoToUrl("https://www.google.com");


        Thread.Sleep(1000000);


    }

Output:

REQ: 71384.640 REQ: 71384.641 REQ: 71384.642 REQ: 71384.643 REQ: 71384.644 REQ: 71384.645 RES: 71384.642 unknown error: unhandled inspector error: {"code":-32000,"message":"No resource with given identifier found"} (Session info: chrome=100.0.4896.75) 71384.642 RES: 71384.643 unknown error: unhandled inspector error: {"code":-32000,"message":"No resource with given identifier found"} (Session info: chrome=100.0.4896.75) 71384.643 RES: 71384.640 unknown error: unhandled inspector error: {"code":-32000,"message":"No resource with given identifier found"} (Session info: chrome=100.0.4896.75) 71384.640 RES: 71384.644 unknown error: unhandled inspector error: {"code":-32000,"message":"No resource with given identifier found"} (Session info: chrome=100.0.4896.75) 71384.644 RES: 71384.641 unknown error: unhandled inspector error: {"code":-32000,"message":"No resource with given identifier found"} (Session info: chrome=100.0.4896.75) 71384.641 RES: 71384.645 unknown error: unhandled inspector error: {"code":-32000,"message":"No resource with given identifier found"} (Session info: chrome=100.0.4896.75) 71384.645

[JamesWilliamsNeve]

Just to add some more context here, seem to have tracked down some weird behaviour that made initial debugging an issue

Firstly this I could never get working:

Dictionary<string, object> cdpParameters = new Dictionary<string, object> { { "requestId", e.RequestId } };
Dictionary<string, object> res = (Dictionary<string, object>)((ChromeDriver)driver).ExecuteCdpCommand("Network.getResponseBody", cdpParameters);

But had luck with the below , taken from the unit tests. No issues here.

   var responseBody = await devToolsSession.Network.GetResponseBody(new GetResponseBodyCommandSettings()
   {
       RequestId = e.RequestId
   });

This also works:

Task.Run(() => { 

   var responseBody = devToolsSession.Network.GetResponseBody(new GetResponseBodyCommandSettings()
   {
       RequestId = e.RequestId
   });
   responceBody.Wait();
   
});

Now the weirdness comes in that it only seem to work if you run it on its own thread. The below will always fail with "no response received" error

var responseBody = devToolsSession.Network.GetResponseBody(new GetResponseBodyCommandSettings()
     {
         RequestId = e.RequestId
     });
responseBody.Wait();

Not sure if that is expected behaviour and I'm misunderstanding something. But not sure what wrapping it up in a task would change.

[johneakin]

Thanks. Using a Task or await when using devToolsSession.Fetch also works. Just had to decode from the base64 string returned to then get the response body.

This all feels so much harder than it needs to be...

[titusfortner]

The better way to do this in c# would require a complete async rewrite, which is under consideration.

Sounds like we have the right answer in the meantime

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.