siteone-crawler icon indicating copy to clipboard operation
siteone-crawler copied to clipboard
verified publisher icon janreges

Feature request: sitemap URL and/or CSV

Open bugnumber9 opened this issue 1 year ago • 1 comments

I think it would be helpful if we could point the crawler to a sitemap URL and/or local CSV file. It's an alternative to the crawler itself trying to discover pages.

bugnumber9 avatar Jan 17 '25 10:01 bugnumber9

You can do so by passing sitemap url to the --url. Not sure if it is included in 1.0.8 (latest release) but in current source it is.

AleksaRistic216 avatar Jan 21 '25 11:01 AleksaRistic216

Hi @bugnumber9,

as mentioned by @DoLoop216, in the current version it is now possible to pass the path to the --url, e.g. to a sitemap. In the next few hours I will try to release version 1.0.9, where this will be included.

janreges avatar Jun 08 '25 13:06 janreges