BadIP, Binary Defense, and Blocklist DE Feeds

[darksheer]

Feeds For:

• badip.com Apache DDoS
• Binary Defense Artillery IP Blocklist
• Blocklist.de: Strong IP
• Blocklist.de: SSH IP
• Blocklist.de: SIP IP
• Blocklist.de: Port 993
• Blocklist.de: Port 80
• Blocklist.de: Port 443
• Blocklist.de: Port 25
• Blocklist.de: Port 22
• Blocklist.de: Port 21
• Blocklist.de: Port 143
• Blocklist.de: Port 110
• Blocklist.de: Mail IP
• Blocklist.de: IRCBOT IP
• Blocklist.de: FTP IP
• Blocklist.de: Bruteforce Login IP
• Blocklist.de: Botnet IP
• Blocklist.de: Apache DDOS
• Blocklist.de: ALL
• Cert Conflicker: Domains
• Chaos Reigns Spam: IP
• CI Army: IP
• Cybersweatshop IMAP: IP
• Cybersweatshop Mail: IP
• Cybersweatshop: Portscan IP
• Cybersweatshop: Ramnode IP
• Greenshow IP
• Hostsfile: GRM Domains
• Hostsfile: HFS Domains
• Hostsfile : HJK Domains
• Hostsfile: MMT Domains
• Hostsfile: Partial Domains
• Hostsfile: PHA Domains
• Hostsfile: PSH Domains
• Hostsfile: PUP Domains
• Hostsfile: Warez Domains
• IPSum: IP Blocklist
• Joewein: SPAM Domains
• Joewein: New SPAM Domains
• Malcode: Blacklist IP
• Malcode: Domains
• Malekal: RSS
• Malshare: Current MD5
• Malshare: Current SHA1
• Malshare: Current SHA256
• Malshare: URL's
• Malware.com: URL's
• Malware Patrol: Dans Guardian URL
• Malware Patrol: Mozilla Adblock URL
• Malware Patrol: Smoothwall URL
• Malware Patrol: Symantec SMTP URL
• Malware Patrol: Symantec Websecurity URL
• Malwareconfig: Full
• Multiproxy: Anon Proxy IP
• Nothink: SNMP Attackers Day IP
• Nothink: SNMP Attackers Week IP
• Nothink: SNMP Attackers Year IP
• Nothink: SSH Attackers Day IP
• Nothink: SSH Attackers Week IP
• Nothink: SSH Attackers Year IP
• Nothink: Telnet Attackers Day IP
• Nothink: Telnet Attackers Week IP
• Nothink: Telnet Attackers Year IP
• Sagadc: Immortal Domains
• Sagadc: Immortal Spyware Domains
• Sagadc: Microsoft Botnet Domains
• Sagadc: Spyware Domains
• Sagadc: Pushdo Botnet Domains
• Sagadc: Zeus Gameover DOmains
• Spysme Proxy IP
• Stop Spam Fourm: 50K Domains
• Stop Spam Fourm: Spam Domains
• Sysctlorg Cameleon Adserver Domains
• yoyo Addservers
• Zeus Tracker: Blocklist Domains
• Zeus Tracker: Blocklist IP
• Zeus Tracker: Blocklist URL Please take a look at the tagging. I tried to replicate what was already outlined, with additions that made sense at the time. Happy to adjust as ncessary.

[darksheer]

In regards to the question about setting an array of URL's to hit, rather than separate feeds for each, I went down the path of separate as I don't fully understand how Yeti displays the feeds to the end user or if the feed jobs would fail entirely.

My understanding based on what I saw was that if one feed were to fail in that configuration, all subsequent feeds would fail as well. Is that correct?

As you pointed out, that also means that the feeds wouldn't run in parallel, which personally I would want them to be independent of any other feeds in case either a code error, a feed is taken down, etc. It also means any adjustments in time would be global across all the matched feeds, where a user would want to modify the interval of one feed, but not the rest.

I guess the tl;dr is, more flexibility, more customization, less impact due to an error (code, feed), easier independent troubleshooting, but that comes with more code.....

I'll get all the other adjustments cleaned up, but would like to hear your thoughts on the less code vs more code+flexability aspect.

[doomedraven]

many of the feeds can be fetched from github, i have the firehol feed, it has all in 1 of them, so we won't process old data over and over

[doomedraven]

https://github.com/yeti-platform/yeti/pull/448

this will add support for firehol, badip.com, binary defense

[doomedraven]

after check all this feeds a lot of them are dead,

the rest uf working feeds, requires check last-modified header

https://lists.blocklist.de/lists/22.txt
http://www.cert.at/static/downloads/data/conficker/all_domains.txt
http://www.chaosreigns.com/iprep/bind_zone.txt
http://charles.the-haleys.org/ssh_dico_attack_hdeny_format.php/hostsdeny.txt
http://cinsscore.com/list/ci-badguys.txt
http://rules.emergingthreats.net/blockrules/compromised-ips.txt
host-file

feed of feeds https://raw.githubusercontent.com/stamparm/ipsum/master/ipsum.txt
http://www.joewein.net/dl/bl/dom-bl-base.txt
http://www.joewein.net/dl/bl/dom-bl.txt
http://malc0de.com/bl/IP_Blacklist.txt

http://www.malshare.com/daily/malshare.current.txt
http://www.malshare.com/daily/malshare.current.sha1.txt
http://www.malshare.com/daily/malshare.current.sha256.txt
http://mirror1.malwaredomains.com/files/immortal_domains.txt
http://www.nothink.org/blacklist/blacklist_snmp_day.txt
http://www.nothink.org/blacklist/blacklist_snmp_week.txt
http://www.nothink.org/blacklist/blacklist_snmp_year.txt
http://www.nothink.org/blacklist/blacklist_ssh_day.txt
http://www.nothink.org/blacklist/blacklist_ssh_day.txt
ttp://www.nothink.org/blacklist/blacklist_ssh_year.txt
http://www.nothink.org/blacklist/blacklist_telnet_day.txt
http://www.nothink.org/blacklist/blacklist_telnet_week.txt
http://www.nothink.org/blacklist/blacklist_telnet_year.txt
http://www.nothink.org/honeypot_telnet_hits.txt

http://dns-bh.sagadc.org/immortal_domains.txt
https://www.stopforumspam.com/downloads/toxic_domains_whole_filtered_50000.txt
https://www.stopforumspam.com/downloads/toxic_domains_whole.txt
https://www.stopforumspam.com/downloads/toxic_domains_whole.txt

https://pgl.yoyo.org/adservers/iplist.php?format=plain&showintro=0

https://zeustracker.abuse.ch/blocklist.php?download=baddomains
https://zeustracker.abuse.ch/blocklist.php?download=badips
https://zeustracker.abuse.ch/blocklist.php?download=compromised

[dumprop]

Any update? BlocklistDe currently implemented hosts file too, but currently it deprecated