renovate icon indicating copy to clipboard operation
renovate copied to clipboard

Published 20 hours ago • verified publisher icon renovatebot

[Feature request] CDKTF Manager

Open secustor opened this issue 1 year ago • 1 comments

Discussed in https://github.com/renovatebot/renovate/discussions/27161

Originally posted by froblesmartin February 8, 2024 New package manager questionnaire at the bottom in collapsible

Hi! I have just started using Renovate, and Dependabot is covering most of our cases, but this is one of those that does not cover.

We have a couple of CDKTF projects, and there we have the Terraform providers and modules defined in the cdktf.json file.

For now, I have managed to update them with a custom manager, but it is not nice. I have to keep the providers as a one-line definition, and the modules as a block, so that they are matched by different regular expressions.

I am considering adding the new manager, but I would need to dive into the project's code. Actually, maybe this would not be a new manager, just an extension to the Terraform one to also support the cdktf.json file and format 🤔 .

I am posting this also so that others can use the custom manager as well for now. 😄

Example cdktf.json:

{
  "terraformProviders": [
    "hashicorp/google@~>5.14.0",
    "hashicorp/google-beta@~>5.14.0",
    "hashicorp/kubernetes@~>2.25.2",
    "hashicorp/helm@~>2.12.1"
  ],
  "terraformModules": [
    {
      "name": "kubernetes-engine",
      "source": "terraform-google-modules/kubernetes-engine/google",
      "version": "~> 30.0"
    },
    {
      "name": "network",
      "source": "terraform-google-modules/network/google",
      "version": "~> 9.0"
    }
  ]
}

And the customManagers:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "enabledManagers": ["custom.regex"],
  "customManagers": [{
    "customType": "regex",
    "fileMatch": ["(^|/)cdktf\\.json$"],
    "matchStrings": [
      "\\s*\\\"(?<packageName>.*?\\/(?<depName>.*?))@~>(?<currentValue>[=>\\s\\w+\\.\\-]*)\\\"?"
    ],
    "datasourceTemplate": "terraform-provider"
  },{
    "customType": "regex",
    "fileMatch": ["(^|/)cdktf\\.json$"],
    "matchStrings": [
      "[\\s\\S]*?\\\"name\\\": \\\"(?<depName>.*?)\\\",[\\s\\S]*?\\\"source\\\": \\\"(?<packageName>.*?)\\\",[\\s\\S]*?\\\"version\\\": \\\"(?<currentValue>[=><~\\s\\w+\\.\\-]*)\\\""
    ],
    "datasourceTemplate": "terraform-module"
  }]
}
New package manager questionnaire

New package manager questionnaire

Did you read our documentation on adding a package manager?

  • [X] I've read the adding a package manager documentation.

Basics

What's the name of the package manager?

CDKTF

What language(s) does this package manager support?

Terraform

How popular is this package manager?

Based on Terraform, becoming more and more popular.

Does this language have other (competing?) package managers?

  • [ ] Yes (give names).
  • [X] No.

What are the big selling points for this package manager?

Supporting a Terraform official Hashicorp project.

Detecting package files

What kind of package files, and names, does this package manager use?

cdktf.json

Which fileMatch pattern(s) should Renovate use?

"fileMatch": ["(^|/)cdktf\.json$"]

Do many users need to extend the fileMatch pattern for custom file names?

  • [ ] Yes, provide details.
  • [X] No.

Is the fileMatch pattern going to get many "false hits" for files that have nothing to do with package management?

No.

Parsing and Extraction

Can package files have "local" links to each other that need to be resolved?

Yes, you can define local modules for example (https://developer.hashicorp.com/terraform/cdktf/create-and-deploy/configuration-file#module-source).

Package file parsing method

Parse the JSON file, and then it is the same as other Terraform files.

The package files should be:

  • [X] Parsed together (in serial).
  • [ ] Parsed independently.

Which format/syntax does the package file use?

  • [X] JSON
  • [ ] TOML
  • [ ] YAML
  • [ ] Custom (explain below)

How should we parse the package files?

  • [ ] Off the shelf parser.
  • [ ] Using regex.
  • [X] Custom-parsed line by line.
  • [ ] Other.

Does the package file have different "types" of dependencies?

  • [X] Yes, production and development dependencies. Providers and modules.
  • [ ] No, all dependencies are treated the same.

List all the sources/syntaxes of dependencies that can be extracted

Same as Terraform

Describe which types of dependencies above are supported and which will be implemented in future

Same as Terraform

Versioning

What versioning scheme does the package file(s) use?

Same as Terraform

Does this versioning scheme support range constraints, like ^1.0.0 or 1.x?

  • [X] Supports range constraints (for example: ^1.0.0 or 1.x), provide details.
  • [ ] No.

Lookup

Is a new datasource required?

  • [ ] Yes, provide details.
  • [X] No.

Will users want (or need to) set a custom host or custom registry for Renovate's lookup?

  • [X] Yes, provide details.
  • [ ] No.

Where can Renovate find the custom host/registry?

  • [ ] No custom host or registry is needed.
  • [X] In the package file(s), provide details.
  • [ ] In some other file inside the repository, provide details.
  • [ ] User needs to configure Renovate where to find the information, provide details.

Are there any constraints in the package files that Renovate should use in the lookup procedure?

  • [ ] Yes, there are constraints on the parent language (for example: supports only Python v3.x), provide details.
  • [ ] Yes, there are constraints on the parent platform (for example: only supports Linux, Windows, etc.), provide details.
  • [X] Yes, some other kind of constraint, provide details.
  • [ ] No constraints.

Will users need the ability to configure language or other constraints using Renovate config?

  • [ ] Yes, provide details.
  • [X] No.

Artifacts

Does the package manager use a lock file or checksum file?

  • [ ] Yes, uses lock file.
  • [ ] Yes, uses checksum file.
  • [ ] Yes, uses lock file and checksum file.
  • [X] No lock file or checksum.

Is the locksum or checksum mandatory?

  • [ ] Yes, locksum is mandatory.
  • [ ] Yes, checksum is mandatory.
  • [ ] Yes, lock file and checksum are mandatory.
  • [ ] No mandatory locksum or checksum.
  • [X] Package manager does not use locksums or checksums.

If lockfiles or checksums are used: what tool and exact commands should Renovate use to update one (or more) package versions in a dependency file?

Package manager cache

Does the package manager use a cache?

  • [X] Yes, provide details.
  • [ ] No.

If the package manager uses a cache, how can Renovate control the cache?

  • [ ] Package manager does not use a cache.
  • [X] Controlled via command line interface, provide details.
  • [ ] Controlled via environment variables, provide details.

Should Renovate keep a cache?

  • [X] Yes, ignore/disable the cache.
  • [ ] No.

Generating a lockfile from scratch

Renovate can perform "lock file maintenance" by getting the package manager to generate a lockfile from scratch. Can the package manager generate a lockfile from scratch?

  • [ ] Yes, explain which command Renovate should use to generate the lockfile.
  • [ ] No, the package manager does not generate a lockfile from scratch.
  • [X] No, the package manager does not use lockfiles.

Other

What else should we know about this package manager?

secustor avatar Feb 09 '24 09:02 secustor

This is a simple and straight forward issue.

  • Follow https://github.com/renovatebot/renovate/blob/main/docs/development/adding-a-package-manager.md
  • Parse the JSON using ZOD via our utilities
  • hashicorp versioning as well as terraform-modules and terraform-providers can be reused.

secustor avatar Feb 09 '24 09:02 secustor