colly

colly copied to clipboard

colly OnError shows EOF error for the given website

4

For my given website after the ```colly.Visit``` function, the ```colly.OnError``` shows **Get "{website}": EOF**. When I check the `StatusCode` it shows `0` **Why?**. Is I'm missing anything? Note: the URL...

kiranupadhyak

Fix require github.com/gocolly/colly/v2: version "latest" invalid: must be of the form v1.2.3

2


youngjuning

Context cancellation

1

Will v3 have context cancellation as a standard part of the API?

pjebs

Switch proxy and revisit on error?

3

Hi I got a use-case, when my proxy-ip got banned because of a big number of req/hour.. I want to remove 1 banned proxy-ip and set a new one... how...

Stormsher

cryptocoinmarketcap crawling example doesn't work.

1

Hi, I'm newbie and I have a question. First of all, thanks to make and share this cooool project! I'm trying to do some examples but the 'cryptocoinmarketcap.go' file doesn't...

dlwogns0128

AllowedDomains and host with port

2

I crawl website like http://127.0.0.1:8080,AllowedDomains is 127.0.0.1:8080,crawl is not work. in code github.com/gocolly/colly/v2.(*Collector).isDomainAllowed at colly.go:752 if d2 == domain { //value is "127.0.0.1" == "127.0.0.1:8080" because line 708,call the parsedURL.Hostname()...

uyid

Hello everyone, I am trying to get an attribute value with colly and I am not being capable of achieve this. `I` I'm trying to get the id value in...

Berveglieri

Fix excessive memory usage

4

When the website has too much text content, the response will take up too much memory. Use sync.Pool to reduce memory applications. Thanks.

x-wesley

Hi, I'm trying to run the most simple colly sample(which goes through the links at go-colly.org) from Goland IDE, but when I try to run it the program just quits....

alenny811

x509: certificate signed by unknown authority

4

Very basic code ` func main() { c := colly.NewCollector() // Find and visit all links c.OnHTML("*", func(e *colly.HTMLElement) { fmt.Println(e) e.Request.Visit(e.Attr("href")) }) c.OnRequest(func(r *colly.Request) { fmt.Println("Visiting", r.URL) }) c.CheckHead...

xjin