a little refactoring on the way to making a pluggable architecture.

[j105rob]

jordan-wright: I am in the process of refactoring to get to a point of a pluggable system for data collection. One thing to note with this pull req is the use of deque to determine what has been processed. Feel free to pull this, as I will be working on this roject over the next couple of days, but if you are not interested, no feelings will be hurt. Thanks for the inspiration and you contribution to the infosec world with this project.

[jordan-wright]

Wow, quite a bit of work done here. Give me some time to review these commits, and I'll see what I can do.

I never mind people working to help make @dumpmon better!

[j105rob]

No problem, still adding stuff. Take your time and ask any questions you may have!!! Cheers!

[jordan-wright]

Wow! There's been some work done here. Good stuff with the dataleak bot!

Sorry it's taken me so long to get back to you. If you're still interested in getting these merged mainline, I'd be happy to make it part of dumpmon!

[jordan-wright]

So, I'll use this to dump some questions I have re: these changes (I hope that's ok!)

Here's the first couple of questions:

• What is the HaveIBeenPaste feed for? Currently, Troy gets the feed by pulling all the pastes from dumpmon's twitter timeline. Wouldn't this make it a bit repetitive?
• I see the code's been refactored to do a regex.search followed by a regex.find_all(). Have you noticed any performance hits by checking each regex against the paste twice?

Comments:

• Pastebin will block your IP if you request more than 600 requests in 10 mins. - Didn't know that! That's interesting and would explain why I kept getting that ban cat :smile:
• I like the idea of making the bot interactive. I've moved away a bit from Mongo to ELK in the mainline bot. I'd still like to keep that Mongo functionality, though.

Really solid stuff. Hopefully I'll find some time soon to start integrating some of these changes into prod.

Thanks for all the great work! This stuff would take dumpmon to the next level, which is exciting.

[j105rob]

Jordan, I'll ping you tomorrow with answers.

We have tons of synergy!

Sent from my iPhone

On Feb 22, 2015, at 6:29 PM, Jordan Wright [email protected] wrote:

So, I'll use this to dump some questions I have re: these changes (I hope that's ok!)

Here's the first couple of questions:

What is the HaveIBeenPaste feed for? Currently, Troy gets the feed by pulling all the pastes from dumpmon's twitter timeline. Wouldn't this make it a bit repetitive?

I see the code's been refactored to do a regex.search followed by a regex.find_all(). Have you noticed any performance hits by checking each regex against the paste twice?

Comments:

Pastebin will block your IP if you request more than 600 requests in 10 mins. - Didn't know that! That's interesting and would explain why I kept getting that ban cat

I like the idea of making the bot interactive. I've moved away a bit from Mongo to ELK in the mainline bot. I'd still like to keep that Mongo functionality, though.

Really solid stuff. Hopefully I'll find some time soon to start integrating some of these changes into prod.

Thanks for all the great work! This stuff would take dumpmon to the next level, which is exciting.

— Reply to this email directly or view it on GitHub.