Collectors engine rework

[Dfte]

Heyo friends!

This PR reworks the entire collectors engine, hope you'll like it!

Changes

Collectors loading

First of all, I removed COLLECTOR dict and all collectors manual imports in entry.py. Instead I replaced them with the following piece of code which dynamically loads files in the donpapi/collectors folders:

def load_collectors(root, collectors_list) -> Tuple[List, List] :
    loaded_collectors = []
    available_collectors = []
    for _, collector_name, _ in iter_modules(path=[f"{root}/collectors/"]):
        available_collectors.append(collector_name)
        if "All" in collectors_list:
            loaded_collectors.append(getattr(import_module(f"collectors.{collector_name}"), collector_name))
        else:
            if collector_name in collectors_list:
                loaded_collectors.append(getattr(import_module(f"collectors.{collector_name}"), collector_name))
    return available_collectors, loaded_collectors

To do so, I had to retrieve the root dirname of donpapi using:

root = os.path.dirname(os.path.realpath(__file__))

This function returns two dicts:

• available_collectors: a list of strings used to build the argparse help menu

group_attacks.add_argument('-c','--collectors', action="store", default="All", help= ", ".join(load_collectors(root, [])[0])+"[...]

• loaded_collectors: the list of loaded collectors that will be run

_, collectors = load_collectors(root, options.collectors.split(","))

Doing so makes PR'ing collectors much easier since all we'll have to do is to add a file in donpapi/collectors/, let's say PowerShellHistory.py and respect the following format:

from typing import Any
from dploot.lib.target import Target
from dploot.lib.smb import DPLootSMBConnection
from donpapi.core import DonPAPICore
from donpapi.lib.logger import DonPAPIAdapter


class PowerShellHistory:
    user_directories = ["\\Users\\{username}\\AppData\\Roaming\\Microsoft\\Windows\\PowerShell\\PSReadLine\\"]

    def __init__(self, target: Target, conn: DPLootSMBConnection, masterkeys: list, options: Any, logger: DonPAPIAdapter, context: DonPAPICore, false_positive: list, max_filesize: int) -> None:
        self.tag = self.__class__.__name__
        self.target = target
        self.conn = conn
        self.masterkeys = masterkeys
        self.options = options
        self.logger = logger
        self.context = context
        self.found = 0
        self.false_positive = false_positive
        self.max_filesize = max_filesize

    def run(self):
        self.logger.display("Defte was here")

Note that the class name MUST be the same as the collector filename (hence PowerShellHistory.py).

Much easier :D

Collector tags

Collector tags were removed for the following:

self.tag = self.__class__.__name__

Which dynamically builds the tag value based on the class name.

We can now call self.logger.secrets the following way:

self.logger.secret(f"[{credential.winuser}] {credential.target} - {credential.username}:{credential.password}", self.tag)

Instead of the old:

self.logger.secret(f"[{credential.winuser}] {credential.target} - {credential.username}:{credential.password}", TAG)

Collectors parameters mutualization

Looking at the collectors, I realised that some of them rely on a false_positive list (there will be more because of the multiple previous PR's I did). Instead of having "magic variables", I mutualized them in the entry.py file:

# Stores the list of false positives usernames:
false_positivee = [
    ".", 
    "..", 
    "desktop.ini", 
    "Public", 
    "Default", 
    "Default User", 
    "All Users", 
    ".NET v4.5", 
    ".NET v4.5 Classic"
]

# Stores the maximum filesize 
max_filesize = 5000000

All collectors inherit these two parameters and as such it will be much more easier to add new false positive users, when found, without struggling modifying all collector files manually.

File dumping

As of now, DonPAPI stores downloaded files creating an entire tree:

This is interesting but I also added the possibility to dump files in a single directory based on the collector name. As such, files can now be found in their respective directories. For example with the NotepadPP and PowerShellHistory collectors:

To do I added a function to the utils.py file:

def dump_file_to_loot_directories(local_filepath: str, file_content: bytes=b"") -> None:
    os.makedirs(os.path.dirname(local_filepath), exist_ok = True)
    with open(local_filepath, "wb") as f:
        f.write(file_content)

Which takes a path and a filecontent and write the file onto the filesystem. This function must be imported in the collectors and can be used the following way:

absolute_local_filepath = path.join(self.context.target_output_dir, *(new_path.split('\\')))
dump_file_to_loot_directories(absolute_local_filepath, file_content)

collector_dir_local_filepath = path.join(self.context.global_output_dir, self.tag, new_path.replace("\\", "_"))
dump_file_to_loot_directories(collector_dir_local_filepath, file_content)

Note that self.context.target_output_dir and self.contexte.global_output_dir are both computed in the core.py file (line 33):

self.global_output_dir = os.path.join(output_dir, DPP_LOOT_DIR_NAME)
self.target_output_dir = os.path.join(output_dir, DPP_LOOT_DIR_NAME, self.host) 
os.makedirs(self.target_output_dir, exist_ok=True)

Before merging this

This PR will have conflicts because of the following ones:

• https://github.com/login-securite/DonPAPI/pull/96
• https://github.com/login-securite/DonPAPI/pull/97
• https://github.com/login-securite/DonPAPI/pull/98
• https://github.com/login-securite/DonPAPI/pull/99

What's next

When merged I'll start adding the following features:

• Pattern matching
• GUI visualization with pattern matching

See ya !!