nolar
Building large operators
Keywords
No response
Problem
I have being looking to do some investigations to writing operators using kopf. From looking at the examples and other projects that use kopf as a dependency, all the kopf code is in a single file. Is it possible to spread this code across multi files? If possible, is there any links to examples that I could explore? Flask blueprints is the style that I am thinking when looking at this problem.
If it is not possible, what was the reasoning behind not allowing it? And is it something that might be on the road map for a future feature?
Hello. Thanks for asking.
Kopf allows using the following CLI:
kopf run -m mod1 -m pkg2.mod2 … file1.py file2.py …
So, the operator can be "assembled" from multiple modules & files.
Besides, Kopf supports embedding the operator into applications and specifying a custom registry (the kopf.OperatorRegistry object) or extending it. The custom registry can be either passed directly to decorators on @kopf.on.somthng(…, registry=…), or set as the default registry via kopf.set_default_registry(…) (before the handler declarations or module imports). And then passed to an embedded operator as an object.
So, there are several ways in which big operators can be assembled from multiple sources and populated dynamically (at startup time, not at runtime).
There is no support for blueprints mostly because of having no sample scenario or practical need for this case.
If that were to be added to Kopf (or outside Kopf), I would try to inherit from kopf.OperatorRegistry and extend it with a list of other operator sub-registries. The only challenge now is that all the execution modules go directly for registry._(watching|changing|spawning|etc)_registry fields, so these fields of the overridden operator should be replaced with proxies that look into the sub-registries instead of holding the handlers directly. Basically, this would implement the idea of blueprints.
Thank you for the answer.
The hopf run option seems like the simple solution to what I want to do but I will explore the the kopf.OperatorRegistry, that seems interesting.
A sample scenario where I could see a blueprint style being usefully is in the go based operator my current team maintains. It is an operator of operators. It handles the configuration of other operator which turn configure their products. This operator of operators product as a whole is installed into different environments that handle sso differently and depending on these environments (installation types) some products are configured differently or not even installed. The project use to have a better example of different installation types but some types have hit EOL and been sense removed.
That's a sample scenario but currently I don't have a practical need for the use case. My current needs are for learning and fun.
I realise the concept of an operator of operators may not be well know. In the articles linked in the kopf documentation the definition of an operator is: controller pattern + API extension + single-app focus. I really like this definition but a single-app can not do much on its own. The operator of operators deliver an application that is built up of multi single-apps.