rod
rod copied to clipboard
go-rod
Missing header in hijack request
When I hijack, the request missing these headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Sec-Fetch-Dest
Currently, we can use this event to get it:
page.EachEvent(func(e *proto.NetworkRequestWillBeSentExtraInfo) {
fmt.Println(e)
})()
After trying test code like this:
rod=cdp go test -v -run Test/TestHijackContinue$
func (s *S) TestHijackContinue() {
url, engine, close := serve()
defer close()
// to simulate a backend server
engine.GET("/", ginHTML(`<html>
<body></body>
<script>
fetch('/a', { headers: { 'My-Header': 'test' } }).then(async (res) => {
document.body.innerText = await res.text()
})
</script></html>`))
engine.GET("/a", ginString(`ok`))
router := s.page.HijackRequests()
defer router.MustStop()
router.MustAdd(url+"/a", func(ctx *rod.Hijack) {
kit.Pause()
})
go router.Run()
s.page.MustSetExtraHeaders("test", "b")
s.page.MustNavigate(url)
kit.Pause()
}
I found it's impossible to get the RequestWillBeSentExtraInfo for a paused request. Unless chrome adds new APIs to support it, we can't do much about it.
I raised an issue for chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=1122782
Related issue: https://github.com/puppeteer/puppeteer/issues/5364#issuecomment-653825110
One hacky solution is to use firefox as the user-agent, because it doesn't support the sec headers yet:
Sample code:
page := browser.MustPage("").MustSetUserAgent(&proto.NetworkSetUserAgentOverride{
UserAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 11.0; rv:82.0) Gecko/20100101 Firefox/82.0",
AcceptLanguage: "en",
})
page.MustNavigate("https://example.com")
Or you can manually set the headers before sending it:
https://github.com/go-rod/rod/blob/3130979a0f746e973fd70b68ddf2ea14e3f7a0c3/hijack_test.go#L41
