gocrawl icon indicating copy to clipboard operation
gocrawl copied to clipboard

Parse <img> tags for crawling, if the option ParseImageTags is set.

Open Kleissner opened this issue 6 years ago • 1 comments

Implement the code for #69 Parsing image tags to include them in crawling. By default the new setting ParseImageTags is set to false, so the current default behavior does not change.

Kleissner avatar Oct 27 '19 10:10 Kleissner

Hello Peter,

Thanks for the PR, makes sense and looks good to me, could you just add a unit test that exercises that new feature (w.opts.ParseImageTags set to true, and checking that image source URLs have actually been added to the crawler)?

Thanks, Martin

mna avatar Oct 27 '19 17:10 mna