webwhiz
webwhiz copied to clipboard
Content fails to be crawled
Hi there, for my new account (created on 2023-10-21), I've setup a first project:
- URL: https://jekyll.one
- Included paths: /pages/public, /posts/public
The crawler for WebWhiz fails completely and no pages were added. A fist crawl seems to work as no issues were reported. When I check the data that has been crawled so far, I got the error message:
Page not found 404
The page you requested could not found. Sorry about that, this shoud not happen.
If I reopen my project, the error message:
Unable to create chatbot. Please try again or contact support
is issued.
I'm using ScreamingFrogSEOSpider to check my site. The crawler of this product reports no issues and all pages gets successfully crawled.
NOTE:: Directory listing is disabled for the Webserver to serve the site.
Thanks for any help, regards Juergen