Check outgoing links: whether they work (no 404), whether the destination uses SSL, whether no bad reputation sites are linked

[kazet]

[kshitijk4poor]

Can be fixed with httpx.

https://github.com/projectdiscovery/httpx

[kazet]

This solves only a part of the task - downloading the content. How would you e.g. solve the reputation problem?

[RasenRhino]

cant we use something like this ?

[kazet]

I think a good first step would be to collect a couple of sites that provide reputation information (and check what they consider bad websites) and only then compare and choose the best one