airy
airy copied to clipboard
airyhq
Api-Installer design proposal
Implementation proposal
The idea is to have a dedicated component that will only handle the installation process of other components and the catalog of available components to install.
This will consist of these endpoints components.install/components.uninstall and components.list. the component will communicate with helm and k8s directly. The two endpoint are described below.
components.install/uninstall
components.list
Great effort @juan-sebastian. As a suggestion in the location of the "components info", I think that we can use the airy-components repo to store the metadata of all the components (open-source, enterprise, community-contributed, ...) and then we can get rid of the "repository" concept. There will be no need to use airy-core/coponent or airy-enterprise/component, but just component.
But the drawback would be if somebody develops his own component that he would like to see in the Catalog - he would need to make a PR to this repo and put the component's metadata there in a separate directory (even though the code can be elsewhere and it also doesn't need to be open source). The component's metadata would have the location of the helm repository and everything needed for the component to run and connect to Airy Core.
@chrismatix @armanjindal correct me if I am wrong, but i think this is why this repo was created?
There is no much effort to support repositories so I think we should keep it. So it will be easier when we implement external repositories. Also there can be cases of we could have airy-core, airy-enterprise and airy-experimental that could have the same component but with different capabilities
@ljupcovangelski Exactly. However that's not a drawback but a feature since we need to formally approve entries into the catalogue. The entire process has been added to our docs by @armanjindal
There is no much effort to support
repositoriesso I think we should keep it. So it will be easier when we implement external repositories. Also there can be cases of we could haveairy-core,airy-enterpriseandairy-experimentalthat could have the samecomponentbut with different capabilities
I think with the new airy-components repo will be good that we will need to approve what is in the Catalog and also this solves us the problem of where to centralize the information about the "components". In that sense - repositories would become obsolete.
Yes this is understood, but I was thinking more in terms of organisation. I mean we can have the same component in different repositories. So it will look something like this.
|-- airy-components
| |-- airy-core
| |-- dialogflow-connector
|. |-- airy-enterprise
| |-- dialogflow-connector
instead of this
|-- airy-components
| |-- core-dialogflow-connector
|. |-- enterprise-dialogflow-connector
@juan-sebastian but then the algorithm to fetch this would have to do a nested loop over directories which would take more network time, right?
@chrismatix not really there are already solution within git API like this. Also this will align with the current setup of helm. We currently install something by providing {repository}/{component}.
ex: airy-core/sources-facebook
If the components are all in one place, I vote for:
|-- airy-components
| |-- facebook-connector
|. |-- dialogflow-connector
| |-- webhook
and then open source or enterprise becomes a property of the component and not a part in the name. Because in the future we might have other types, such as third party or proprietary.
Also when I proposed the repository concept, the motivation was that a whole repository is added with all of its components and that we don't need to have control over what is in the repository. But as a component would need to be approved by Airy in order to be listed in the Catalog - then this isn't that important any more.
couldn't agree more on @ljupcovangelski opinion!
|-- airy-components
| |-- facebook-connector
|. |-- dialogflow-connector
| |-- webhook
thank you for outlining the proposal @juan-sebastian 🙏
On my end, I really like @ljupcovangelski's proposal of having all the components in one place and putting the repository as a component's property. This way, the components' naming is stable (which benefits the frontend) and a component's 'repository' is still present in the info.
I agree with @ljupcovangelski. @juan-sebastian, it is true we need users to know which is which, but I think the filtering of enterprise and core can be done on the front end of the catalog. From an engineering point of view, there is no reason to distinguish these from each other into two separate repositories since the way we install them is the same. This has the added benefit of standardizing the naming, which @AudreyKj pointed out.
One more comment @juan-sebastian about the name. As the api-installer is doing install, uninstall and list, maybe api-components is a more suitable name? And the configuration of the components can be api-configuration.
I find those two names a bit strange (as configuration is also about the components) so I would suggest that we have them both as separate microservices, but under the same component airy-components. And we can have then two deployments:
- api-components-installer
- api-components-configuration
but under the same component/helm-chart -
api-components.
Similar to how we have for the sources (sources-facebook-event-router and sources-facebook-connector are both under sources-facebook). And anyway it will be strange to separate them (to have only the api-installer without the api-configuration feature).
I agree with @ljupcovangelski that handling components is one concern so it should be one logical component. Given this, the name of the pods doesn't matter that much so what is proposed looks good to me.
