crawl4ai
crawl4ai copied to clipboard
[Bug]: CrawlResult Links not taking into account base tag
crawl4ai version
0.6.3
Expected Behavior
The links array in CrawlResult should be derived based on if there is a base tag on the page.
should navigate to http://xyz.com/abc.pdf
Current Behavior
It current does not take into account the base tag and generates an href relative to the current page
Is this reproducible?
Yes
Inputs Causing the Bug
Steps to Reproduce
Code snippets
OS
All
Python version
3.11
Browser
Chrome
Browser version
No response
Error logs & Screenshots (if applicable)
No response