Enhance export decorator to support meta-methods

[DissectBot]

TBD: AC and scope. Implementation requirement is basically the POC? Think if this can be broken down to smaller steps.

Problem statement: Need to improve the defining method for scraping plugins.

Current syntax disadvantages:

• Prefixing with e.g. scraped_ is clumsy
• No clear programmatic distinction between regular and scrape plugins
• Scrape plugins only work on fully working targets - defeats the purpose of scraping
• Limiting new features creation

Solution proposal:

Improving the existing @export decorator will enable (at least):

1. An easy and ergonomic way for plugin developers to add scraping functionality to their plugin
2. Ability for plugins to provide “progress” feedback
3. Allow other methods to be “hard-linked” to a specific plugin function

In short, I propose a export decorator that makes the following syntax possible, inspired by the @property decorator.

class FooPlugin(Plugin):
    @export
    def bar(self):
        ...
        
    @bar.progress
    def bar(self, ...):
        ...
        
    @bar.scrape
    def bar(self, ...):
        ... 
        
    @export
    def baz(self):
        ...
        
    @baz.scrape
    def baz(self, ...):
        ... 
        

Benefits of this syntax:

• quite “Pythonic” and easy to understand for plugin writers
• providing hard language binds to the respective “source” method (bar and baz)
• allowing for any number of “meta-methods” to be added to a plugin method
• exposing progress bars feature for target-query enabled by detailed information as provided by a plugin (e.g. give a hint or exact number of expected records), or a generic “records per second” style fallback.

All the @<func>.scrape methods could additionally receive a basic set of arguments that will be generic over all scraping plugins, such as whether to scrape encrypted or LVM volumes, or only free or slack space. These arguments can be easily forwarded into the scraping helpers and passed in by tools like target-scrape.

Minimal POC to demonstrate how this syntax works

Some of the magic in this snippet explained in a few sentences:

• The meta class and special dictionary are needed to register each “meta-method” as a unique member of the class. We need this to piggyback for method binding
• The Export decorator returns a custom binded MethodType here to allow for sub-attribute access. The sub-attribute access is generalized to retrieve that from the binding class (this is where we piggy back the method binding)
• These two magics are not strictly necessary if we just store everything directly in the Export itself, and bind manually in the __getattr__ of the custom MethodType. However a benefit of the meta class is that getattr(ExamplePlugin, "foo.scrape") works as expected too.
• There may be a small performance hit on method calling because of the custom MethodType, but I don’t believe this to be a major problem as you generally only call exported methods once.
• If going with a “decorator class” like this POC, we can also move most “function attributes” we currently set with our decorators to this class.

import types


class MetaDict(dict):
    def __setitem__(self, key, value):
        if getattr(value, "__meta__", False):
            key = value.__name__
        return super().__setitem__(key, value)


class PluginMeta(type):
    @classmethod
    def __prepare__(cls, name, bases, **kwargs):
        return MetaDict()


class Plugin(metaclass=PluginMeta):
    pass


class MethodType:
    def __init__(self, func, obj):
        self.__method__ = types.MethodType(func, obj)

    def __str__(self) -> str:
        return str(self.__method__)

    def __repr__(self) -> str:
        return repr(self.__method__)

    def __call__(self, *args, **kwargs):
        return self.__method__(*args, **kwargs)

    def __getattr__(self, name):
        if not name.startswith("__"):
            key = f"{self.__method__.__func__.__name__}.{name}"
            return getattr(self.__method__.__self__, key)

        return getattr(self.__method__, name)


class Export:
    def __init__(self, func):
        self.func = func

    def __get__(self, instance, owner):
        if instance is None:
            return self
        return MethodType(self.func, instance)

    def __set_name__(self, owner, name):
        self.__name__ = name

    def _meta(self, func, meta):
        func.__name__ = f"{func.__name__}.{meta}"
        func.__qualname__ = f"{func.__qualname__}.{meta}"
        func.__meta__ = meta
        return func

    def scrape(self, func):
        return self._meta(func, "scrape")

    def progress(self, func):
        return self._meta(func, "progress")


def export(*args, **kwargs):
    def decorator(func):
        return Export(func)

    if len(args) == 1:
        return decorator(args[0])
    return decorator


class ExamplePlugin(Plugin):
    @export
    def foo(self):
        print("foo export")

    @foo.scrape
    def foo(self):
        print("foo.scrape")

    @foo.progress
    def foo(self):
        print("foo.progress")