Discussison: bus instance for each skill?

[forslund]

Chatting with @JarbasAI last night and discussing the multithreaded event emitter in the messagebus client and the fact we don't guarantee the execution order of the different threads.

@JarbasAI suggested that each skill should have a separate bus client, and run in single threaded mode.

The pros and cons

Running in this manner will keep responsiveness high while being a bit safer and more isolated. In addition the startup of skills can be simplified a bit.

The downside of using a bus client per skill is that it

• Requires more memory (but only slightly)
• Using the current design it will require 1 thread per skill

A possible issue may be heavy increased number of connections to the messagebus service but I currently haven't seen any signs of this.

I've been a bit against this setup for above reasons but after talking with @JarbasAI and thinking it over I think it's positive effects will outweigh the negatives...

The client has always been more or less threaded to keep responsiveness high (even before the ExecuterEventEmitter and ThreadedEventEmitter) but always using a threadpool with fairly low thread count.

Proof of concept

To test this out I created this feature/bus-client-per-skill-branch depending on the feature/custom-emitter-branch of the messagebus-client.

This will give skills their own bus client using the basic event-emitter (no threads). The executor event emitter is still used by the services, since there was some annoying blocking behavior. Consider this mainly as a POC of how it would work, in the end it should be possible to use only the single-threaded approach.

Things to note is the behavior of the common play / common query multicasts, in this setup they are as quick as when using the ExecutorEventEmitter.

We might find a way around using the 1 thread per skill by taking advantage of asyncio but I'm not 100% sure how / if it would work.

[krisgesling]

Hey, going to take some time to think about this - but I can certainly see the benefits.

I'd be interested to see how a system with lots of Skills goes. Eg if we're talking about higher resource usage, what does that look like in practice? Is it completely linear with the number of Skills loaded?

[ChanceNCounter]

This stems from a sequence of "what ifs" that (probably) most immediately stemmed either from a revisit of Jarbas' Evil Skill, or from one of the times I whined about the difficulty of preventing malicious listeners or loggers in some implementations.

The realization of this concept in HiveMind happened in pieces. First, and I know I keep saying this, we stated that other assistants have less trouble with this issue because other assistants are 1) part of their respective operating systems, and, therefore, 2) expect other software to integrate with the assistant, rather than integrating the assistant with other software.

Separately came the obvious fact that HiveMind clients are not running the Skills to which they respond. Those are on a Mycroft device somewhere else on the Hive. That means we'll need to implement something that can translate a Mycroft bus message into an instruction the Hive client can execute - and that the client is under no obligation to speak only Mycroft-language. Having done that, and owing to the fact that Hives are nestable, we arrived at a place where "complex" Hive clients can run HiveMind-core without Mycroft, and have clients of their own: your phone is a client of your "main" Mycroft device, and the integrated apps on your phone are each, separately, individual clients of the client on your phone. You talk to the phone, the phone talks to Mycroft, Mycroft sends information to the app, via the phone, encrypted with the app's keys. Not even the phone's "primary" Mycroft client can see what apps see in the clear.

Whew.

Moving Skills themselves to the same paradigm was the next logical step, but I can't speak with total confidence to the performance implications because we never really thought about them with respect to the Hive. Hive-integrated apps will only need the websocket and HiveMind protocol layers, and enough Mycroft lib to provide the Mycroft protocol layer.

However, the bus itself is an async websocket server. A websocket client is cheap. It's the multiple-instantiation of certain Mycroft packages that might hurt. If MycroftAI is interested, I am prepared to upstream some of the modular stuff that makes this so lightweight for us, but this would ultimately represent a design choice which it may not be possible to walk back, so it's not something I'm about to sort on a lark.

(There's probably a way to do it right in Mycroft-core anyway.)

[JarbasAl]

like @ChanceNCounter said there is a hivemind component that loads skills in isolation https://github.com/JarbasHiveMind/LocalHive