Support a way to issue admin commands via a local socket?

[harlowja]

During discussion (and agreement) that openstack would move its various bots (based on various python bot frameworks and code and such) @ http://lists.openstack.org/pipermail/openstack-dev/2016-July/thread.html#100409 one of the outcomes of that discussion was the the current configuration mechanism that is used to reload plugins and such is not really desired to be based on private IRC (or other backend) commands from a certain user to the bot.

The folks there (that will manage these plugins and the associated bot) have the following ideas (discussion for this is @ http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-07-29.log.html#t2016-07-29T18:17:00) or desires.

• A way to configure the bot without having to go through IRC private commands to tell a plugin how to configure itself (it seems http://errbot.io/en/latest/user_guide/provisioning.html allows for using storage as a way to configure a plugin, so this seems doable) and this will match (and/or match close enough) how the folks there use configuration management to describe how there plugins are configured & work.
• Instead of admin commands being issued over IRC via a certain user (or set of users) a desire to be able to trigger those commands via a private-application socket (or SIGHUP?) that the previously mentioned configuration management can also use to tell plugins to reload (after that configuration management writes out there new config).

The first one there seems do-able with what exists, the second one seems like it could require some work(?), although it seems like the 'text' backend could be easily used to create a private-application socket backend that is used for this (so that there would be a public backend, and a private 'text' backend that is reachable via a local app socket).

[gbin]

Glad to hear that openstack is moving to Errbot !

I though a little bit about your point 2 over the weekend.

What about using the webhook feature for this, porting the features of https://github.com/errbotio/errbot/blob/master/errbot/core_plugins/plugins.py ?

(untested pseudocode to get the idea).

Should work by posting a {'name': 'myplugin'} to /reload_plugin:

    @webhook
    def reload_plugin(self, payload):
        if 'name' not in payload:
             return 'ERROR: malformed request'
        name = payload['name']
        if name not in self._bot.plugin_manager.get_all_plugin_names():
            return ("{} isn't a valid plugin name. The current plugins are:\n"
                   "{}".format(name, self.formatted_plugin_list(active_only=False)))
        try:
            self._bot.plugin_manager.reload_plugin_by_name(name)
            return "Plugin %s reloaded." % name
        except PluginActivationException as pae:
            return 'Error activating plugin %s: %s' % (name, pae)

[harlowja]

That might work, is that restricted to who can call those webhooks? I'd rather not have someone who gets the IP address of the bot (not a hard thing to get via a whois equivalent in IRC?) be able to trigger that reload (maliciously or not)? That's the nice thing about a local 'admin' socket is that its pretty hard to maliciously via some external actor trigger that.

[gbin]

The auth part:

You can use webhooks with a server-side SSL certificate + retrieve auth headers from the raw request:

https://github.com/errbotio/errbot/blob/master/docs/user_guide/plugin_development/webhooks.rst#the-raw-request.

Alternatively, I think a better solution would be to use client side certificates with a small nginx frontend, it has also the advantage of beeing flexible for the other public webhooks you implement (throttling etc, ...).

The local socket:

So unfortunately rocket doesn't seem to support local sockets, the closest thing would be a localhost one 127.x.x.x. rocket is basically unmaintained at that point so it might push us to come back to something like Flask that now support py3.

[harlowja]

So why wouldn't just a local backend that runs the text backend work? Would running a private to the app (local socket) backend and a public backend be that hard?

Something @ https://github.com/errbotio/errbot/blob/master/errbot/bootstrap.py#L122 that takes a backend like "text+irc" or "slack+irc" (the latter "slack+irc" doesn't yet need to happen, but would be nice).

What would happen if a thread was spun up for each backend there? Would they share the same plugins, or perhaps we can just have a tiny new backend, call it 'admin' that responds to a few selected local commands (and that's it); maybe its not a backend at that point but just a tiny class that is setup (as a thread) in https://github.com/errbotio/errbot/blob/master/errbot/bootstrap.py#L122

Something like:

try:
        bot = backendpm.get_plugin_by_name(backend_name)
        bot.attach_storage_plugin(storage_plugin)
        bot.attach_repo_manager(repo_manager)
        bot.attach_plugin_manager(botpm)
        bot.initialize_backend_storage()
        bot.initialize_admin_port()
    except Exception:
        log.exception("Unable to load or configure the backend.")
exit(-1)

[harlowja]

Any further thoughts here?

[gbin]

So it goes back to have a multibackend support: it would require a profound refactoring and would make Errbot a chat "bridge". I think while backends are maturing and have a cool stable API across we might try out one day to make this bridge, but for "feature creep" reasons it would be a separate product (imagine all the bloat about mapping identities between backends etc...).

Maybe making a simple socket on a thread on a plugin might be better if you really want to go the socket way: you will have access to the full bot from self._bot from the plugin so you can hook into the bot like:

        msg = Message(text_from_the_socket)
        msg.frm = self.build_identifier(self._bot.config['ADMINS'][0])  # pick any admin so it is recognized as an admin
        msg.to = self._bot.identifier  # direct message to the bot 
        self._bot.callback_message(msg)  # trigger the message ingestion

That should be all you need to do to inject an admin message from a socket buffer.

[zoni]

@gbin how would you feel about offering an optional way to override the configuration of plugins via config.py?

I'm envisioning something like this:

# Default, no override:
CONFIG_OVERRIDE = None

# Override configuration for the webserver plugin:
CONFIG_OVERRIDE = {
    'Webserver': 
        {'HOST': '0.0.0.0',
         'PORT': 3141,
         'SSL': {'certificate': '',
                 'enabled': False,
                 'host': '0.0.0.0',
                 'key': '',
                 'port': 3142
        }
}

This would allow people to set configuration from config.py which can easily be managed via a configuration management system such as puppet, chef, etc. Doing this is a lot easier to manage than doing it via errbot --storage-set, errbot --storage-merge, etc.

Going through the openstack discussions referenced in the original issue, I believe that if we offered this, we would have everything to satisfy their use-case (it's already possible to restrict/disable the option of on-the-fly (un)loading of plugins or changing their config via ACLs).

I'm aware you're more in the camp of dynamically configuring errbot via chat command but I see the value of a more static way to configure it via static configuration files.

[sijis]

I realize i'm going back to an older unresolved discussion but i thought I thought it would be beneficial to add some additional notes for future folks searching a similar issue.

Before closing this out, I just wanted to point out a few things that are now possible which could help.

• External storage and a series of storage plugins - https://github.com/search?q=err-storage
• Contributor websocket plugin - https://github.com/attakei/errbot-backend-webapp
• Provisioning via the cli (previously mentioned) - https://errbot.readthedocs.io/en/6.1.1/user_guide/provisioning.html