gocrawl

gocrawl copied to clipboard

Apply for submitting A Chineses version of markdown

1

Hello, I'm interested in gocrawl and I want to submit Chinese version of translation of readme. What can I do ? Can I give a PR on that?

favany

Parse image tags

2

The crawler should optionally parse image tags that are in the form `` and use it as input for crawling (just as it would consider `` links to images). I'm...

Kleissner

Graceful & Proper Termination, both internally and via Extender

2

The current stop functionality via `Crawler.Stop` is insufficient for multiple reasons: * Calling `Stop` twice results in a panic (because it would close `c.stop` twice) * Inside the functions `Extender.Visit`...

Kleissner

Implement ReadLimit to optionally limit reading from response body #71

1

Re #71

Kleissner

Parse <img> tags for crawling, if the option ParseImageTags is set.

1

Implement the code for #69 Parsing image tags to include them in crawling. By default the new setting `ParseImageTags` is set to false, so the current default behavior does not...

Kleissner

There should be an optional read limit for reading the response body - otherwise a website could literally respond with GBs of data. The problem was already raised (and incorrectly...

Kleissner