Support GitLab CI/CD components

[secustor]

Discussed in https://github.com/renovatebot/renovate/discussions/23424

^{Originally posted by fgreinacher July 18, 2023}

Tell us more.

CI/CD components are a new way to reuse CI logic within GitLab, see upstream documentation.

They look very similar to other includes and can probably be implemented in the existing gitlabci-include datasource.

include:
  - component: gitlab.com/gitlab-org/[email protected]

[viceice]

this should be added to the gitlabci manager, as the gitlab-include manager will be merged to it

[fgreinacher]

I started having a look at this. I think with the current syntax it might be challenging to implement: https://gitlab.com/groups/gitlab-org/-/epics/9897#note_1494759770.

[thompson-shaun]

Does the directory structure update in 16.3 make the project/component distinction firm enough to move forward? Reference Merge Request.

[mbrodala]

Does someone maybe already have a working (read: tested) custom manager for this which could be used in the meantime?

[lorenzogrv]

I'm researching about GitLab CI/CD components. I'm curious about why the include syntax is challenging. My understanding about:

include:
   - component: $CI_SERVER_HOST/$CI_PROJECT_PATH/$EXAMPLE@$REF

where:

• $CI_SERVER_HOST points to GitLab instance, i.e. gitlab.com
• $CI_PROJECT_PATH points to namespace within domain, i.e. iaigz/gitlab-ci
• $EXAMPLE references to component name - i.e. lint-yaml-files
• $REF references to repository history - tag, branch, or commit sha

Within the component definitions themselves, the YAML syntax is the usual Gitlab CI syntax, preceded with more YAML front-matter. any other gitlabci-include can contain that spec: syntax too, up from GitLab 16.x, as it's usable from include:local

---
spec:
  inputs:
    job-stage:
      default: test
    search-path:
      default: .
    search-name:
      default: Dockerfile
---
lint-dockerfiles:
  image: hadolint/hadolint:v2.12.0-alpine
  stage: $[[ inputs.job-stage ]]
  script:
    - hadolint --version
    - find $[[ inputs.search-path ]] -type f -name $[[ inputs.search-name ]]
      -exec hadolint
      '{}'
      \+

The key part seems matching for the - component: <host><repo><depName>@<version> pattern. Note that version can't be specific to one component within the repository, but the same to every component for any given git reference

[lorenzogrv]

I'm experimenting with the example at https://docs.renovatebot.com/modules/manager/regex/#using-custommanager-to-update-the-dependency-name-in-addition-to-version, it may fit.

[fgreinacher]

I'm curious about why the include syntax is challenging.

@lorenzogrv In previous iterations this syntax supported components in a any subdirectory, making it impossible to extract the project path from the reference.

Your understanding is correct and it should be doable now.

[benedikt-bartscher]

@lorenzogrv @mbrodala Here is my custom manager for cicd components:

{
  "customType": "regex",
  "fileMatch": [
      "\\.gitlab-ci\\.ya?ml$",
      "templates/.+\\.ya?ml$"
  ],
  "versioningTemplate": "semver",
  "datasourceTemplate": "gitlab-releases",
  "registryUrlTemplate": "https://{{{registryUrl}}}",
  "depTypeTemplate": "repository",
  "matchStrings": [
    "component: (?<registryUrl>[^\\s\\/]+)\\/(?<depName>\\S+)\\/[^\\s\\/]+@(?<currentValue>\\S+)"
  ]
}

[mbrodala]

@benedikt-bartscher Thanks, seems to work just fine! :+1:

I only noticed that gitlab-releases are slow to pick up updates, so one may consider using gitlab-tags as datasourceTemplate instead.

[benedikt-bartscher]

@mbrodala you are welcome. That's interesting, i will investigate why gitlab-releases is slower later this evening. Do you still get release notes in your renovate MRs with gitlab-tags as datasource?

[mbrodala]

Yes, this works just fine. Renovate retrieves release notes separately from the specific data source AFAIK.

[lorenzogrv]

@benedikt-bartscher Works like a charm, thanks for sharing. I used gitlab-tags as suggested by @mbrodala and release notes are actually included within MRs.

[mbrodala]

Finally, thanks @fgreinacher !