lazynlp
lazynlp copied to clipboard
urllib fails without headers
Hi, Thanks for this great tool.
I noticed urllib fails with a Forbidden Request
error when I call download_page
on some links. You can reproduce the error by trying the code below:
import lazynlp
link = "https://punchng.com/"
page = lazynlp.download_page(link, context=None, timeout=None)
This raises a 403 as shown below.
I've attempted to create a PR that adds headers to the request by default.