wget2 icon indicating copy to clipboard operation
wget2 copied to clipboard

Unable to pass without errors

Open Zyles opened this issue 8 months ago • 0 comments

My quest continues to get this running using ansible.

It keeps failing the ansible build because the crawl returns an error:

PLAY RECAP **********************************************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Which is a blocker for the ansible build to complete.

I am using this:

$ wget2 --recursive --level 100 --no-hsts --adjust-extension --span-hosts --content-disposition --trust-server-names --cut-file-get-vars --exclude-directories=/wp-content,/r --domains sub.domain.com --no-parent sub.domain.com

What happens is, it reports a 404 error on some urls because the website has some broken urls.

I want to completely ignore/skip crawling content in /wp-content and /r so I can return a crawl without errors.

Or at least not exit with an error code even if it finds 404.

Is this possible?

Zyles avatar Jun 03 '24 09:06 Zyles