[🐛 Bug]: Using the NetworkManager.NetworkResponseReceived in Selenium Chromedriver breaks auth http proxy extensions

[strich]

What happened?

I need to use a http proxy with auth in Selenium and AFAIK the only way to do this is via a custom extension at the moment. I have successfully tested this and it works fine (code below).

However it seems to fail if I attach a network listener like so:

NetworkManager manager = new NetworkManager(Driver);
manager.NetworkResponseReceived += NetworkResponseReceived;
await manager.StartMonitoring();

Whenever that code runs, all browsing attempts return a 407 unauthorized from I assume the proxy. Its as if the NetworkManager listener is maybe eating/intercepting the extension listener?

Custom extension:

    class ProxyExtension
    {
        private string _dir;
        private string manifestJson = @"
{
    ""version"": ""1.0.0"",
    ""manifest_version"": 2,
    ""name"": ""Chrome Proxy"",
    ""permissions"": [
        ""proxy"",
        ""tabs"",
        ""unlimitedStorage"",
        ""storage"",
        ""<all_urls>"",
        ""webRequest"",
        ""webRequestBlocking""
    ],
    ""background"": {""scripts"": [""background.js""]},
    ""minimum_chrome_version"": ""76.0.0""
}
";

        private string backgroundJsTemplate = @"
var config = {{
    mode: ""fixed_servers"",
    rules: {{
        singleProxy: {{
            scheme: ""http"",
            host: ""{0}"",
            port: {1}
        }},
        bypassList: [""localhost""]
    }}
}};

chrome.proxy.settings.set({{value: config, scope: ""regular""}}, function() {{}});

function callbackFn(details) {{
    return {{
        authCredentials: {{
            username: ""{2}"",
            password: ""{3}""
        }}
    }};
}}

chrome.webRequest.onAuthRequired.addListener(
    callbackFn,
    {{ urls: [""<all_urls>""] }},
    ['blocking']
);
";

        public ProxyExtension(string host, int port, string user, string password)
        {
            _dir = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName());
            Directory.CreateDirectory(_dir);

            string manifestFile = Path.Combine(_dir, "manifest.json");
            File.WriteAllText(manifestFile, manifestJson);

            string backgroundJs = string.Format(backgroundJsTemplate, host, port, user, password);
            string backgroundFile = Path.Combine(_dir, "background.js");
            File.WriteAllText(backgroundFile, backgroundJs);
        }

        public string DirectoryPath
        {
            get { return _dir; }
        }

        ~ProxyExtension()
        {
            Directory.Delete(_dir, true);
        }
    }

How can we reproduce the issue?

Use `--load-extension` after instanciating the ProxyExtension class and providing it the DirectoryPath. When you launch chrome the extension should be enabled and working. Try running with and without the NetworkManager interception.

Relevant log output

None.

Operating System

Windows 11

Selenium version

4.16.2

What are the browser(s) and version(s) where you see this issue?

ChromeDriver 1192607

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver 1192607

Are you using Selenium Grid?

No response

[github-actions[bot]]

@strich, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[github-actions[bot]]

Hi, @strich. Please follow the issue template, we need more information to reproduce the issue.

Either a complete code snippet and URL/HTML (if more than one file is needed, provide a GitHub repo and instructions to run the code), the specific versions used, or a more detailed description to help us understand the issue.

Note: If you cannot share your code and URL/HTML, any complete code snippet and URL/HTML that reproduces the issue is good enough.

Reply to this issue when all information is provided, thank you.

[strich]

All the necessary code is there. Please let me know if anything is missing.

[diemol]

How can someone copy and paste that code to reproduce the issue?

[strich]

How can someone copy and paste that code to reproduce the issue?

As a Selenium contributor I'm sure you have a million templates of creating a chromedriver instance and going to www.google.com. Do you really need me to go make a repro project for that? The only unique thing in this ticket, in addition to the code in the ticket, should be that you'll need a http proxy with user:pass auth to test with.

[diemol]

How can someone copy and paste that code to reproduce the issue?

As a Selenium contributor I'm sure you have a million templates of creating a chromedriver instance and going to www.google.com. Do you really need me to go make a repro project for that? The only unique thing in this ticket, in addition to the code in the ticket, should be that you'll need a http proxy with user:pass auth to test with.

Actually, I don't. We are a few volunteers with paid jobs as our priority. So my intention is to leave this issue in a state where someone can jump and try to reproduce it with the least amount of work. I understand that creating a whole solution to report your issue might be annoying, but while doing that you are helping us help you fix the issue.

[titusfortner]

I can't shed light on how the custom extension might conflict with network interception, but there is another way to do this — you can have a mitm proxy. Send the location of that proxy to the driver, and then have that proxy add authentication to send to the proxy that controls access to your network. That would keep the network things separate at least.

I haven't tried it, but I think this tool should work: https://github.com/saucelabs/forwarder

[diemol]

I am closing this because we did not get more information.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.