How to hijack requests under proxy

[dhlebin]

Rod Version: v0.100.0

Hello, I am trying to use HijackRequests under proxy that requires auth, but page doesn`t loading

l := launcher.New().Proxy("111.11.111.11:8888").MustLaunch()
browser := rod.New().ControlURL(l).Trace(true).MustConnect()
go browser.MustHandleAuth("login", "password")()

router := browser.HijackRequests()
router.MustAdd("*.jpeg", func(hijack *rod.Hijack) {
	hijack.ContinueRequest(&proto.FetchContinueRequest{})
})
go router.Run()

page := stealth.MustPage(browser)
page.Timeout(time.Minute).MustNavigate("https://google.com").MustWaitLoad()

Result:

[rod] 2021/07/02 00:10:32 [wait] load <page:299D146C>
panic: context deadline exceeded

What am I doing wrong? please help

[Bolado]

You can use my reply to have an idea on how to work with proxies that require auth: https://github.com/go-rod/rod/issues/305#issuecomment-743352401

And then use the transport of the proxy like this: ctx.LoadResponse(&httpclienttogen, true) https://github.com/go-rod/rod/issues/189#issuecomment-727282118

[ysmood]

Might be a chrome bug, you can use Request.SetClient to use your own custom proxy, you don't have to use Proxy("111.11.111.11:8888"). If you use Request.SetClient then it will turn into a common Go issue, just google how to set HTTP proxy in Go.

[dhlebin]

Thanks for your reply, my main goal is to figure out can I use browser.MustHandleAuth together with browser.HijackRequests or it will not work?

[ysmood]

@dhlebin I think this is a limitation of chrome: https://github.com/puppeteer/puppeteer/issues/3253

[dhlebin]

Ok, thank you!

[Bolado]

Thanks for your reply, my main goal is to figure out can I use browser.MustHandleAuth together with browser.HijackRequests or it will not work?

Well, like I said on my previous reply , first you need to learn how to make a http client with auth, and then use LoadResponse with it

The example code:

httpClient := http.Client{}
var proxyStr string
var auth string
var proxySorted []string

proxySorted = strings.Split(proxy, ":")

switch ProxyType {
case "http":
	proxyStr = ("http://" + proxySorted[0] + ":" + proxySorted[1])
case "https":
	proxyStr = ("https://" + proxySorted[0] + ":" + proxySorted[1])
case "socks":
	proxyStr = ("socks://" + proxySorted[0] + ":" + proxySorted[1])
}
switch ProxyType {
case "http", "https":
	proxyURL, _ := url.Parse(proxyStr)
	
	if len(proxySorted) > 2 {
		auth = proxySorted[2] + ":" + proxySorted[3]
		basicAuth := "Basic " + base64.StdEncoding.EncodeToString([]byte(auth))
		hdr := http.Header{}
		hdr.Add("Proxy-Authorization", basicAuth)
		transport := &http.Transport{
			Proxy:              http.ProxyURL(proxyURL),
			ProxyConnectHeader: hdr,
			IdleConnTimeout:    20 * time.Second,
		}
		httpClient.Transport = transport
	} else {
		transport := &http.Transport{
			Proxy: http.ProxyURL(proxyURL),
		}
		httpClient.Transport = transport
	}
}
ctx.LoadResponse(&httpClient, true)