[ci-visibility] Create deployment correlate subcommand

[rodrigo-roca]

What and why?

This PR creates a new subcommand called datadog-ci deployment correlate which will upload some data into a public API to correlate CD deployments and pipelines.

This command is targeted for gitOps CD providers like argocd that use git repositories as a source of truth. In these cases the customers usually have 2 repositories: a source code repository where the business logic is and a configuration repository with a bunch of configuration files (most likely k8s YAMLs). The end goal of this command is to map source code commits to configuration commits. We also need to gather some environment variables so we can identify the pipeline more accurately.

As you can see on the updated README file, it has to be called in a very specific point in the CI for it to work properly. At that point the git information from the git commands and the environment variables are not the same:

• The git information in the environment variables is from the source code repo
• The git information obtained from running commands comes from the configuration repo since it is called after making some commits to it.

The requests expected by the API have the following format:

{
    "data": {
        "type": "ci_app_deployment_correlate",
        "attributes": {
            "cd_provider": "argocd",
            "ci_provider": "gitlab",
            "config_repo_url": "https://github.com/my-k8s-config-repo",
            "config_commit_shas": [
                "eaca0b7e862622e5eba4b14dbfac7bae85af22a5"
            ],
            "ci_env": {
                "git.commit.sha": "faca0b7e862622e5eba4b14dbfac7bae85af22a6",
                "git.repository_url": "https://github.com/DataDog/my-source-code-repo",
                "CI_PIPELINE_ID": "1",
                "CI_JOB_ID": "1"
            }
        }
    }
}

In the example request above the CI_PIPELINE_ID is an example environment variable from gitlab but the endpoint supports the environment variables from the other supported providers as well.

How?

A brief description of implementation details of this PR.

Review checklist

• [x] Feature or bugfix MUST have appropriate tests (unit, integration)

[datadog-datadog-prod-us1[bot]]

Datadog Report

Branch report: rodrigo.roca/create-deployment-correlate-command Commit report: 712f979 Test service: datadog-ci-tests

:white_check_mark: 0 Failed, 2869 Passed, 0 Skipped, 2m 3.95s Total duration (59.8s time saved)

[rodrigo-roca]

Can you follow this section and tell me if anything is missing/unclear? Thank you 😁

All seemed clear to me 👍 Thanks for pointing this out

[Drarig29]

~~@rodrigo-roca the codeowners have to be updated as well 😁~~

[rodrigo-roca]

@Drarig29 The codeowners already encapsulates the folder no? The command is datadog-ci deployment correlate so everything is inside the "deployment" folder.

src/commands/deployment    @DataDog/ci-app-libraries

[Drarig29]

Oh indeed, my bad 🤦. I'll reflect this in the docs 👍

Edit: added in https://github.com/DataDog/datadog-ci/commit/c73045d59f2be583bfd1ecdf886eeb481076571d