colly
colly copied to clipboard
gocolly
Elegant Scraper and Crawler Framework for Golang
[scraper-gocolly1.txt](https://github.com/gocolly/colly/files/9297917/scraper-gocolly1.txt) A program of mine, which worked for over a week now, just out of the nowhere stopped working in the sense that one html element as specified div#gs_res_ccl_mid cant...
The code accidentally panics in a certain OnResponse, how can the code not stop? Let other requests continue.
I'm trying to scrap [https://dnsdumpster.com](https://dnsdumpster.com), and to scrap exact values I need to pass few headers when requesting, but colly doesn't seems like having support for custom headers yet, only...
I use the huge proxies to crawl amazon website, but in onerror function, i can not get the response.Request.Proxy Url because it was not always display. I want to remove...
This PR adds [ClusterfuzzLite](https://google.github.io/clusterfuzzlite/) and a fuzzer for the HTML unmarshalling. ClusterfuzzLite will run fuzzers in the CI when PRs are made. It can be extended beyond this PR with...
My first three or four tests were ok and there were no problems until the next test had problems the entry_data field which is the most important field in Instagram...
I know this sounds weird, but I'm looking at a website that has no div names, and so on, in a nutshell, a nightmare. With this in mind, I found...
I am trying to get the TLS certificate that a site is presenting during the TLS handshake. I looked through the documentation and the response object but did not find...
**the example:** ``` func main() { url := "https://httpbin.org/delay/1" // Instantiate default collector c := colly.NewCollector(colly.AllowURLRevisit()) // create a request queue with 2 consumer threads q, _ := queue.New( 2,...
