dub
dub copied to clipboard
Link Scaning: Domain + URL Scanning to Flag Suspicious / Malicious Links
In this PR, I've used the Pangea's URL and Domain Intel that scans malicious links using datasets from Crowdstrike and Domaintools.
Since detecting malicious URLs is hard to get 100% right with just one dataset, I first do a broad domain intel lookup to see if a domain is suspicious using the domain intel API powered by the Domaintools dataset. Then, if it is suspicious, I proceed to perform a URL intel lookup using the URL intel API powered by the CrowdStrike dataset. The best way to scan for malicious URLs is to use multiple dataset providers as implemented in this case.
To use the APIs, you will need to create an account on Pangea and enable URL Intel
and Domain Intel
services on the same PANGEA_TOKEN
. Then paste the PANGEA_TOKEN
and PANGEA_DOMAIN
into the .env
file.
@steven-tey Not sure if you wanted to keep the favicon URL lookup, so I removed it for now 😅