[Feature] Create a terraform provider for devlake

[felipem1210]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Use case

As a DevOps engineer I would like to configure Devlake using terraform, so that way I don't have to spend a lot of time using the UI to configure projects/blueprints/webhooks/connections/transformations-rule .

Description

The advantages of Terraform are always following the philosophy of InfraAsCode. We can benefit DevLake project with this to provide a way to configure faster devlake without doing a mess. By inheriting the configuration of Devlake through, we can create automation, that will narrow the number down of unmanaged resources to zero and everything will be managed through a repository, providing power to other people to contribute and be independent more efficiently.

When creating a terraform provider, it has to be stored in a new repository, treated as a separate project, the provider needs to be uploaded to the terraform public registry https://registry.terraform.io/ and then it needs to be maintained, in order to add new resources (DevLake components as: projects, blueprints, webhooks, connections, etc), and to correct bugs.

The user that wants to use the provider needs to add it to its terraform files in order to be able to create all its devlake configuration with this IaC tool.

To get it done we would need to have contributors interested to maintain this repo. I propose myself and @hugolesta as first contributors/maintainers, but any other golang developer can contribute. Here is showed the initial guideline to begin the project

We would like to begin to work on this project under the apache organization (if possible) if not, we can begin it on one of our personal workspaces. Here is an example of a terraform provider made for another apache solution: kafka

Related issues

No response

Are you willing to submit a PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[klesh]

Looking forward to the PR ❤️

[Startrekzky]

Awesome. If I understand correctly, with this feature implemented, DevLake configuration can be done entirely through Terraform, but the sync process management (e.g., viewing blueprint's sync history, status, etc.) is still done through the Config UI?

[hugolesta]

Hello @Startrekzky, exactly, the terraform provider will have resources that represent configurations for DevLake (Config UI), therefore, any configuration that is defined in this way would be possible to replicate in an uncountable number of Devlake solutions.

BTW those configurations would be stored in a terraform backend, meaning the changes will not be lost during the time as long the backend is protected by the team/organization.

I assume this kind of feature increases automation, increasing the delivery speed in too many teams and organizations worldwide.

As @felipem1210 said, we need a separate repository to start working on this, we are not sure if the DevLake community will provide this repository or if it can be handled by us in a different repository.

[d4x1]

There is a project https://github.com/apache/incubator-devlake-helm-chart , so I think creating a seperat repository is acceptable.

[Startrekzky]

@hugolesta I think adding another repo is acceptable. I just have to confirm the process with other PPMCs and our mentors. I'll get back to you once it's set up.

[JorgeGar]

@Startrekzky @hugolesta I think having a separate repo would be the way to go in this case. The terraform module would potentially have a separate lifecycle, so it would be cleaner, in my opinion, to keep them separated.

[klesh]

Hi, All, I've created a repo for the purpose, https://github.com/apache/incubator-devlake-terraform

[felipem1210]

Hi @klesh . Thanks for creating the repo. After talking with @hugolesta we concluded that we will need full admin permission to the repo, to be able to setup branch protection rules, manage secrets for workflows, etc.. Can you provide us with the permission? Also the name of the repository needs to be changed. It has to be terraform-devlake-provider

Thanks

[klesh]

@felipem1210 I don't think full admin permission is possible because all repos belong to the Apache Foundation while I can only create repos for the Apache DevLake project, and the name must be prefixed with incubator-devlake-, so , terraform-devlake-provider is not possible. However, it is possible to set up the repo via the .asf.yaml file, would you like to check if it is sufficient for the purpose?

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.

[Startrekzky]

Hi @felipem1210 @JorgeGar , are there any updates on this issue or the permission of the terraform repo?

[github-actions[bot]]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.