gospider
gospider copied to clipboard
Proxy settings not fully honored
It seems that proxies are not honored, by looking at Wireshark traffic I see some requests not going through any proxy.
I think this is related to https://github.com/gocolly/colly/issues/392
We probably need to disable keep alives like:
c.WithTransport(&http.Transport{
DisableKeepAlives: true,
})
in the code here