Support Terraform as a workload to deploy and orchestrate cloud resources

[hongchaodeng]

There are many users who rely on Terraform or similar tooling (ARM, ROS) to deploy infrastructure resources on cloud, e.g. EC2, RDS, k8s cluster. After deploying infra resources, users want to use OAM on top of Terraform-based infrastructure as the application model for separating concerns and unifying k8s native architecture.

Since Terraform is a widely used, popular open source tool, we should model Terraform as a Workload type and provide spec and examples to onboard more Terraform users to OAM seamlessly.

[resouer]

Good idea.

Though, I think Terraform is more like a runtime implementation for OAM than simple extension workload type. To me, Terraform is K8s for infrastructures. So the goal would be Terraform users could have an option to deliver app to oam-terraform with app centric spec.

[AdrienFromToulouse]

👋 @hongchaodeng I Wanted to jump into this issue because I am not sure to precisely understand.

I may still not be familiar enough with the Workload notion (I apology in advance), but I have hard time to relate the usage of Terraform itself and the Workload concept.

You mean if the workload is of type Mysql then it maps directly to a Terraform provider able to deploy a Mysql? But in that case I would say it is up to the workflow engine like Argo to trigger the Terraform according to the dependency Graph isn't it? That's what @resouer kind of said?... :/

That is how we started to see how we could use OAM as standard layer for downstream deployment orchestration. Like a container app deployed to k8s thanks to rudr and if workflow is of type mysql then run a Terraform routine first before pushing to rudr.

In our case we need to validate the OAM schema first and then rudr in particular will still double check because of its admission controller.

Am I completely off track?

Thanks in advance for your clarifications :)

[resouer]

@AdrienFromToulouse Yes it's close to what I am thinking.

Orchestration system like Terraform would be more like a runtime implementation, as OAM concepts (Component, Trait, Scope etc) will be mapped to multiple parts of Terraform. So combine them together, it's an application atop Terraform. The simplest implementation is "API converter "(OAM t yaml -> Terraform yaml), should work :-)

The workload, as you asked, is the way to answer "how to run this component" in OAM. It should be mostly mapped to instance provisioned by Terraform like a container, or a database instance. On the other hand, there're capabilities of Terraform should be mapped to Traits, workflow management is definitely one of them.

[resouer]

I deep dived Terraform examples for a bit, here's a draft plan:

1. A generic controller impl, i.e.: https://github.com/spotahome/gontroller;
   1. or we just install the controller in a minimized K8s (it's simplest).
2. Use (1) to implement a OAM to Terraform converter and translate controller
   1. we need to decide which provider we want to play first. Or, let's say, three? (AWS, Azure, Alibaba)
   2. a one-time converter to generate OAM component spec, for example, terraform.alibabacloud.oam.dev/v1beta2.ecs, from existing .tf resource
      1. or we should manually do this work?
   3. a oam-terraform controller to translate oam spec to .tf and provision corresponding resources.

For application, we should use dockerized app as a start.

And we will leave trait and scope as TODO list and use workload as first PoC.

[wonderflow]

For 1, I think it's better to install the controller in a minimized k8s, that would be easier.

For 2, I totally agree!

[ghostbody]

I'm curious about a question that can we implement an OAM runtime without k8s, or say the "controller pattern" ? 🤔

[resouer]

@ghostbody Implementing OAM without k8s is even simpler because you don't need to care about controller pattern at all. Just translate the OAM yamls into Terraform or CloudFormation files is enough. This can even all be done in client side. For example, the AWS's work: https://github.com/awslabs/amazon-ecs-for-open-application-model

I also proposed another server side solution here (which is more "fancy"): https://github.com/oam-dev/spec/issues/245#issuecomment-554842672. But honestly speaking, if I can choose the translate solution, why not? :-)

[ghostbody]

I take a look at amazon-ecs-for-open-application-model. It's a proof of concept project which translates the OAM spec to CloudFormation. It partially support the OAM spec and voilate some guidelines of OAM (for example, component is immutable in OAM but not in AWS).

To be more deeply, I think there is a big difference between k8s solution and Terraform or CloudFormatio solution.

• The spec of the OAM is designed based on k8s api object which is originally fits k8s
• The k8s implementation is acutally a infrastructure as data pattern, while Terraform or CloudFormatio is a infrastructure as code pattern. They are different ideas. infrastructure as data is better for upgrade situation.
• The k8s runtime solution kubevela is quit simple. You just need to register CRDs and controllers, then "throw" api objects to k8s and k8s will make all the things done. However, implementation without k8s need a lot of extra work.
• The descriptive capability of OAM and Terraform or CloudFormatio maybe not the same which means solution like Terraform or CloudFormatio can not meet all spec requirements.

[resouer]

@ghostbody Yes, your understanding about IaD vs IaC is correct. So the approaches available to make Terraform K8s native are:

1. Crossplane with Terraform as provider: https://github.com/crossplane/crossplane/pull/1677/files
2. Use Terraform operators, for example: https://github.com/kubeform/kubeform

The output of both approaches are CRDs represent cloud resources provisioned by Terraform, which then could be referenced by OAM/KubeVela as Components to deploy. If you would like to explore this direction, please kindly let us know! We are very happy to have a community project support this scenario.