spidr
spidr copied to clipboard
Respect base tags
Currently <base href="...">
tags are not taken into account and will send the spider to the wrong URL on pages with a base tag. With this patch, the spider correctly calculates absolute URLs when a base tag is present.