web-scraper-chrome-extension icon indicating copy to clipboard operation
web-scraper-chrome-extension copied to clipboard

Include input sitemap URLs in output

Open chrisyoungr opened this issue 7 years ago • 5 comments

@jwillmer do you know how I can include my input sitemap URLs into the output upon scraping so it's clear what output matches with each input? I'm looking for the correct area to tweak your code.

chrisyoungr avatar Jan 31 '18 13:01 chrisyoungr

It is a while back since I invested time in this project and I did not write it from scratch - just extended it. So I can't really tell you at the moment. But I will fix an open bug in the next two weeks. Maybe I can give you advice after that if it is still needed.

jwillmer avatar Jan 31 '18 16:01 jwillmer

That'd be great. Thanks @jwillmer !

chrisyoungr avatar Jan 31 '18 17:01 chrisyoungr

@jwillmer were you able to resolve the bug and have any new ideas on this. It's a struggle to use without the input info being included

chrisyoungr avatar Feb 23 '18 17:02 chrisyoungr

Have a look at this functions. I think you could include your code into the last one.

https://github.com/jwillmer/web-scraper-chrome-extension/blob/cba6351392f0bf84d9eb06e6bb78a5371efb4396/extension/scripts/Controller.js#L1124

https://github.com/jwillmer/web-scraper-chrome-extension/blob/cba6351392f0bf84d9eb06e6bb78a5371efb4396/extension/scripts/Controller.js#L1089

https://github.com/jwillmer/web-scraper-chrome-extension/blob/cba6351392f0bf84d9eb06e6bb78a5371efb4396/extension/scripts/Controller.js#L1046

jwillmer avatar Feb 23 '18 21:02 jwillmer

@jwillmer I've looked into this but am still struggling with getting it to successfully include the original start URLs to the output. If you have any suggestions, I'd greatly appreciate it!

chrisyoungr avatar Apr 19 '18 20:04 chrisyoungr