Harpoon V2

[Te-k]

Need a major rewrite to:

• Have a cleaner data model and interface
• Handle separately getting data and presenting data (allowing to have default views like JSON)
• Keep a history of data acquired
• Allow integration with other tools?
• Have IP info as default tools for plugins to use
• Use click

It needs some thinking but this first version is very messy

[anarcat]

totally unsollicited feedback here, but here we go. ;)

not a fan of click. tries to do too many things and i quickly hit limitations on others. it's quite a big pill to swallow, i prefer plain argparse nowadays.

one thing that's blocking me in adopting harpoon wholesale (including writing plugins, crowdsec would be nice, for example) is the lack of a batch mode...

right now i found that i can do harpoon intel $IP and that cycles through all (?) plugins looking for dirt on the given IP... but sometimes what i have is dozens (if not hundreds) of IPs hammering my logs and I want to answer questions like:

• are all those hits from the same host or network? (count hits per IP, CIDR, ASN, asncount in harpoontools does part of this)
• is this part of a botnet? (shodan? crowdsec?)
• is this all on the same network? (ASN lookups, traceroutes?)
• is this a tor exit node? or on some other RBL? (crowdsec, spamhaus, dronebl, etc)

I would like to feed a plain text file listing IPs (maybe sorted, but not deduplicated aka not uniq(1)) and have harpoon just "do the right thing"...

Right now I use this as a very crude approximation of part of the first one:

whois -h whois.cymru.com "-v $(sort -u ips-20231106)" | sort -r | sort -ru

all the rest is manual lookups, one IP at a time. so batch lookups would be very important for me...

[Te-k]

I totally welcome feedback on Harpoon :)

Indeed, I tried to move to click but creating commands based on plugins seemed way more complex than argparse so argparse will stay.

The intel plugin cycles through all plugin that have intel enabled in the configuration file (doc is also needed for v2 :D).

I am not sure exactly how to implement what you say. Would you like one command for all these questions? It is possible to just implement intel by reading in a file with output in JSON, would that help? Or having different commands for each one?

Also if you don't know it yet, Greynoise looks like a great addition to your toolkit

[anarcat]

On 2023-11-09 15:30:13, Tek wrote:

I totally welcome feedback on Harpoon :)

Indeed, I tried to move to click but creating commands based on plugins seemed way more complex than argparse so argparse will stay.

The intel plugin cycles through all plugin that have intel enabled in the configuration file (doc is also needed for v2 :D).

I am not sure exactly how to implement what you say. Would you like one command for all these questions? It is possible to just implement intel by reading in a file with output in JSON, would that help? Or having different commands for each one?

Also if you don't know it yet, Greynoise looks like a great addition to your toolkit

Thanks for the feedback!

I think one command that takes a list of IPs would be nice. Something simple: just one IP per line is fine, even better if it counts the number of duplicates so you get a hit rate profile.

Here the "intel" command doesn't fire up virustotal, even though it has intel: true in the config, not sure what's up with that...