dialoqbase icon indicating copy to clipboard operation
dialoqbase copied to clipboard

Published 20 hours ago • verified publisher icon n4ze3m

Bug: Crawler scrapes page anchors as new pages

Open chriswessels opened this issue 1 year ago • 1 comments
trafficstars

When using the Crawler, if there are any links in a page that point to an anchor (e.g. href="#anchor"), these are treated as new Webpages and scraped separately. Links that resolve to anchors on the same page should not resolve to new pages.

chriswessels avatar Jan 15 '24 11:01 chriswessels

Yes, the crawler is currently stable; I will fix it soon.

n4ze3m avatar Jan 15 '24 12:01 n4ze3m