std icon indicating copy to clipboard operation
std copied to clipboard
verified publisher icon divnix

Integrate OAM as a 4th layer action that also covers delivery

Open blaggacao opened this issue 3 years ago • 7 comments

https://oam.dev/

Based on this hint, I realized that this is a super useful abstraction to very effectively quell the vendor-specificity of the 4th layer.

/cc @disassembler @manveru @nrdxp

blaggacao avatar Oct 05 '22 02:10 blaggacao

I don't think OAM, as it currently stands, is something that can be trivially implemented in an agnostic way. Unlike what the authors (presumably, Alibaba) propose, the model is not completely independent:

  1. It seems to assume the existence of a control plane for managing the application lifecycle (referred to as a runtime). In the specification, this seems to be deceptively omitted, yet can easily be inferred to be Kubernetes.
  2. It heavily implements the same YAML-based API schema used by Kubernetes. For example, many definitions infer an apiVersion and kind which, in most cases, directly refers to Kubernetes types.
  3. It seems to exclusively favor "cloud native" applications
  4. The upstream project, which is solely based around Kubernetes, is feeding back into the specification. How they plan on keeping the spec unbiased isn't clear (and reading the issues, it appears they are not concerned about it).

The first point is key: throughout the specification, the "OAM runtime" is referenced as the entity which is responsible for realizing the OAM configuration. In the case of the authors, this (currently) exclusively refers to Kubernetes.

The second point is particularly alarming, considering new tools are being introduced almost daily to address the madness that is colloquially referred to as "YAML hell." It appears this project isn't immune, see here for an example of trying to reference values across definitions.

That being said, I still think there's a lot that can be learned from the model to further our own understanding. As a short review:

  • Application: The primary building block. It consolidates the artifact(s), configuration, and delivery requirements into a single entity. It is produced by a cooperative effort between both dev and ops.
  • Component: A deployable artifact. It's an artifact in that it usually consists of some sort of runnable and it's deployable in that it includes information about what type of workload can "run" it. A component is defined via a "schematic" which is what is ultimately used by platforms to "deploy" the component. A schematic must accept one or more parameters that can be used to tune the component. The output of the schematic is usually tightly coupled with the workload (for example, a k8s deployment workload type would expect a k8s deployment definition).
  • Workload: A workload is a platform-specific implementation for deploying/running components. The primary purpose of a workload is to define how a component is deployed and what traits a component is allowed to have. Examples of types of workloads include containers, deployments, databases, etc.
  • Traits: A component may have one or more traits that are specified by the type of workload associated with the component. A trait is independent of both a component and workload, however, a trait must explicitly define what types of workload it is allowed to be associated with (this is enforced by the OAM runtime). A trait is dependent on the OAM runtime, as it's the runtime that explicitly performs whatever operations necessary to ensure the characteristics of a trait are met (i.e. configuring SSL for an SSL trait).
  • Application Scopes: Sits between the component and application layer. Provides a loose way to couple components together inside of an application. For example, a network scope can group several components into a single network boundary as defined by the underlying platform.

Relating this to the current model being used by std is a bit difficult. A component seems to consist of an operable and a more abstract version of layer 3. In other words, layer 3 most closely resembles the "workload" part of a component, however, the OAM model doesn't distinguish it as an OCI image.

A component also seems to bring in layer 4 as well. The output of a component (the schema) is directly consumed by the OAM runtime and used in conjunction with the workload type to deploy the component to an environment. For example, in a Kubernetes runtime, a component may use a deployment workload and output a deployment object which is then consumed and executed by the Kubernetes runtime.

There appears to be no direct correlation between traits/scopes to anything described in std.

All this is to say that the OAM specification seems to be incompatible with the current paradigms being used by std. It's not possible to cleanly relate layers 1-4 proposed by std with what OAM is expecting. It would require a paradigm shift to more closely align with what OAM suggests as the layers of an application (application/component).

jmgilman avatar Oct 05 '22 18:10 jmgilman

First thanks for your deep insights, are really valuable for me.

I'm very agree with you there is a k8s smell all around the project. I also have detected some designs you analyzed have changed a lot in KubeVela side.

For instance workloads and scopes were de-facto deprecated. Now all implementation lays into ComponentDefinitions. Also policy and workflow were introduced. AFAIK the main places to find last improvements or good samples are:

  • https://github.com/kubevela/samples I feel interesting the Terraform_DEMO one.
  • https://github.com/kubevela/kubevela/tree/master/design/vela-core This include an Appfile proposal with less k8s reminiscences.
  • https://github.com/kubevela/kubevela/tree/master/vela-templates/definitions CUE implementations of different definitions.

About std layers, of course I only imagine OAM as a 4th layer previous to render to different schedulers.

oneingan avatar Oct 05 '22 21:10 oneingan

Thanks for your response. I knew there were some deprecations, but I didn't realize how much had been deprecated. The terraform example still uses workload, though. A lot of the other examples still seem to use it as well. Was this a recent deprecation? It'd be nice to see an example that documents the newer schema of components.

The divergence between the OAM specification and Kubevella is also slightly worrying. It seems one is iterating at an order of magnitude faster than the other. I think I would prefer to wait for greater maturity before adopting this model as a maintenance point for std.

For the std integration to work well, it's likely we'd need to be doing a lot of generation (nix -> yaml). If major portions of the API landscape are still getting deprecated, it's going to create a lot of turbulence for downstream consumers. I'd imagine this is also currently a deterrent for other "runtimes" being developed.

jmgilman avatar Oct 05 '22 22:10 jmgilman

I understand:

  • OAM should use Nickel (and if they don't, we should [or Nix in the mean time])
  • Standard would implement a OAM-ish runtime (better word: OAM control plane / OAM controller). The nature of layer 4 (or what comes beyond) is not suitable for stateless operation or for abusing git for state. → stdd
  • Layer 3 targets distribution, so I guess we're free to choose (currently OCI, but semantically not limited to). So I kind of agree with @uningan that we're dealing with sort of an enriched 4th layer.
  • The OAM's / Kubevela's glorified rendering stuff is nonsense (and just a symptom of "YAML hell"), we have IFD, if plain Nix/Nickel functions are not capable enough.
  • The most valuable aspect is the workflow implementation / remote state machine. No CI/CD platform currently fits the bill for the environment propagation business process. I'd favor an intergration point with BPMN2.0 / zeebe to tackle that "business-y" part of the SDLC with the appropriate tool / modeler. Since it's a trait, the OAM runtime should just be able to schedule them onto a BPMN runtime such as zeebe(/cc @GTrunSec).

EDIT:

Note that this comment hint stuff is functionally equivalent to the Standard Registry under __std:

        // +vela:cli:enabled=true
        // +vela:cli:usage=specify commands to run in container
        // +vela:cli:short=c

blaggacao avatar Oct 06 '22 01:10 blaggacao

Standard would implement a OAM-ish runtime (better word: OAM control plane / OAM controller).

I'm still trying to piece together exactly what runtime is in the context of OAM. I think it might be synonymous with "scheduler", in which case we'd have to pick a scheduler to start with and potentially have multiple runtimes to support other schedulers. Unless you're suggesting we create our own scheduler :)

jmgilman avatar Oct 06 '22 04:10 jmgilman

Thanks for your response. I knew there were some deprecations, but I didn't realize how much had been deprecated. The terraform example still uses workload, though. A lot of the other examples still seem to use it as well. Was this a recent deprecation? It'd be nice to see an example that documents the newer schema of components.

Sure, official kubevela doc shows workload attribute is only used to model certain platform capabilities but can be omitted: https://kubevela.io/docs/platform-engineers/oam/x-definition

oneingan avatar Oct 06 '22 06:10 oneingan

I'm still trying to understand OAM/KubeVela well, I'll summarize some key points I've discovered:

  • The only public interface is Application yaml with their list of Components, Traits, Policies and Workflow.
  • "OAM itself has no enforcement on how to implement the schematic as long as it could: a) model a deployable unit; b) expose a JSON schema or equivalent parameter list". Source: https://github.com/oam-dev/spec/blob/master/3.component_model.md#schematic
  • They are considering a simplified Application interface: https://github.com/kubevela/kubevela/blob/master/design/vela-core/appfile-design.md
  • Plan to "Support build application from source code for developer and use OCI infrastructure to delivery application." Source: https://kubevela.io/docs/roadmap/2022-12-roadmap

About std role:

  • I'm very agree about Nickel as rendering engine.
  • I have no strong ideas about the Control Plane matter, and how std could fill the gap here. But maybe could bundle full workflow/toolchain (as in https://porter.sh) + control loop logic and run in whatever scheduler user choose.

Edit: Something similar in Porter roadmap: https://github.com/getporter/porter/projects/4#card-52677173 Edit2: Porter just released v1.0.0 and they'll focus in a Porter Operator (https://getporter.org/blog/v1-is-here/).

Disclosure: I'm not hardly positionated favor OAM, but I'm developing a Devops suite using std and if it have "pseudo-standard" interface will ease the acceptation in non-Nix store.

oneingan avatar Oct 06 '22 10:10 oneingan

We cae leave this out for now ad even remove from the backlog so that we can focus on more near-term features / todos

blaggacao avatar Feb 18 '23 02:02 blaggacao