Pull mirroring not being enabled on first apply, needs to apply again

[igorbrites]

GitLab Provider version

3.15.0

GitLab version

GitLab Enterprise Edition 15.1.0-pre 53a8011dea2

Terraform version

❯ terraform -v
Terraform v1.2.2 on darwin_arm64
+ provider registry.terraform.io/gitlabhq/gitlab v3.15.0

Relevant Terraform Configuration

locals {
  has_issues    = !var.is_template
  has_downloads = !var.is_template
  has_projects  = !var.is_template
  has_wiki      = !var.is_template

  github_mirror_url = "https://${var.github_user}:${var.github_token}@github.com/${github_repository.default.full_name}.git"
}

resource "github_repository" "default" {
  name        = var.name
  description = var.description

  visibility  = var.visibility
  is_template = var.is_template

  has_issues    = local.has_issues
  has_downloads = local.has_downloads
  has_projects  = local.has_projects
  has_wiki      = local.has_wiki

  allow_rebase_merge = false
  allow_merge_commit = false

  delete_branch_on_merge = true
  auto_init              = true
  vulnerability_alerts   = true

  dynamic "template" {
    for_each = var.template != null ? [var.template] : []

    content {
      owner      = template.value.owner
      repository = template.value.repository
    }
  }
}

resource "github_team_repository" "default" {
  count = length(var.teams)

  team_id    = var.teams[count.index]
  repository = github_repository.default.name
  permission = "admin"
}

resource "gitlab_project" "default" {
  count = var.gitlab_namespace_id == null ? 0 : 1

  name             = var.name
  description      = var.description
  visibility_level = var.visibility
  namespace_id     = var.gitlab_namespace_id
  lfs_enabled      = false

  import_url                          = local.github_mirror_url
  mirror                              = true
  mirror_trigger_builds               = true
  mirror_overwrites_diverged_branches = true
  only_mirror_protected_branches      = false

  provisioner "local-exec" {
    command = "curl -X DELETE -H \"PRIVATE-TOKEN:$GITLAB_TOKEN\" https://gitlab.com/api/v4/projects/${self.id}/protected_branches/${github_repository.default.default_branch}"
  }
}

resource "gitlab_service_github" "default" {
  count = var.gitlab_namespace_id == null ? 0 : 1

  project        = gitlab_project.default[0].id
  token          = var.github_token
  repository_url = github_repository.default.http_clone_url
  static_context = true
}

Relevant log output

❯ terraform apply
Running apply in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/...

Waiting for the plan to start...

Terraform v1.2.2
on linux_amd64
Initializing plugins and modules...
github_membership.igorbrites_membership: Refreshing state... [id=<REDACTED>]
github_team.devops_team: Refreshing state... [id=<REDACTED>]
module.nodejs_template.github_repository.default: Refreshing state... [id=nodejs-template]
github_team_membership.igorbrites_membership: Refreshing state... [id=<REDACTED>]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.nodejs_template.gitlab_project.default[0] will be created
  + resource "gitlab_project" "default" {
      # All default and non-related values removed
      + import_url                                       = (sensitive) # Here goes the GitHub token in the URL
      + mirror                                           = true
      + mirror_overwrites_diverged_branches              = true
      + mirror_trigger_builds                            = true
      + name                                             = "nodejs-template"
      + namespace_id                                     = <REDACTED>
    }

  # module.nodejs_template.gitlab_service_github.default[0] will be created
  + resource "gitlab_service_github" "default" {
      + active         = (known after apply)
      + created_at     = (known after apply)
      + id             = (known after apply)
      + project        = (known after apply)
      + repository_url = "https://github.com/<REDACTED>/nodejs-template.git"
      + static_context = true
      + title          = (known after apply)
      + token          = (sensitive value)
      + updated_at     = (known after apply)
    }

Plan: 2 to add, 0 to change, 0 to destroy.


Do you want to perform these actions in workspace "github-repositories"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.nodejs_template.gitlab_project.default[0]: Creating...
module.nodejs_template.gitlab_project.default[0]: Provisioning with 'local-exec'... # Local exec to remove main branch protection
module.nodejs_template.gitlab_project.default[0] (local-exec): Executing: ["/bin/sh" "-c" "curl -X DELETE -H \"PRIVATE-TOKEN:$GITLAB_TOKEN\" https://gitlab.com/api/v4/projects/36770535/protected_branches/main"]
module.nodejs_template.gitlab_project.default[0] (local-exec):   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
module.nodejs_template.gitlab_project.default[0] (local-exec):                                  Dload  Upload   Total   Spent    Left  Speed
module.nodejs_template.gitlab_project.default[0] (local-exec):   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
module.nodejs_template.gitlab_project.default[0] (local-exec):   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
module.nodejs_template.gitlab_project.default[0]: Creation complete after 5s [id=36770535]
module.nodejs_template.gitlab_service_github.default[0]: Creating...
module.nodejs_template.gitlab_service_github.default[0]: Creation complete after 1s [id=122606327]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

❯ terraform apply
Running apply in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/...

Waiting for the plan to start...

Terraform v1.2.2
on linux_amd64
Initializing plugins and modules...
github_membership.igorbrites_membership: Refreshing state... [id=<REDACTED>]
github_team.devops_team: Refreshing state... [id=<REDACTED>]
module.nodejs_template.github_repository.default: Refreshing state... [id=nodejs-template]
github_team_membership.igorbrites_membership: Refreshing state... [id=<REDACTED>]
module.nodejs_template.gitlab_project.default[0]: Refreshing state... [id=<REDACTED>]
module.nodejs_template.gitlab_service_github.default[0]: Refreshing state... [id=<REDACTED>]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.nodejs_template.gitlab_project.default[0] will be updated in-place
  ~ resource "gitlab_project" "default" {
        id                                               = "<REDACTED>"
      ~ mirror                                           = false -> true
      ~ mirror_overwrites_diverged_branches              = false -> true
      ~ mirror_trigger_builds                            = false -> true
        name                                             = "nodejs-template"
        tags                                             = []
        # (65 unchanged attributes hidden)


        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.


Do you want to perform these actions in workspace "github-repositories"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.nodejs_template.gitlab_project.default[0]: Modifying... [id=<REDACTED>]
module.nodejs_template.gitlab_project.default[0]: Modifications complete after 1s [id=<REDACTED>]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Description

When creating a new project on GitLab.com by importing it from GitHub (or anywhere else), the mirroring is not applied, needing to apply again to make it work.

After first apply:

After second apply:

[timofurrer]

Thanks for the report @igorbrites 👍

I've managed to reproduce this using the following minimal terraform config:

resource "gitlab_project" "default" {
  name = "issue-1119"

  import_url                          = "https://github.com/timofurrer/test.git"
  mirror                              = true
  mirror_trigger_builds               = true
  mirror_overwrites_diverged_branches = true
  only_mirror_protected_branches      = false
}

.. but only on gitlab.com without a premium license (you need one for the mirror feature as documented here). The above code works fine on my local GitLab EE (Ultimate License) 15.0.

In addition I can also reproduce it using the API directly:

project=$(curl -X POST "https://gitlab.com/api/v4/projects" --header "PRIVATE-TOKEN: $GITLAB_TOKEN" --header "Content-Type: application/json" --data '
{
 "name": "issue-1119",
 "visibility": "public",
 "import_url": "https://github.com/timofurrer/test.git",
 "mirror": true,
 "mirror_trigger_builds": true
}
' )

project_id=$(echo "$project" | jq -r '.id')

curl -X GET "https://gitlab.com/api/v4/projects/$project_id" --header "PRIVATE-TOKEN: $GITLAB_TOKEN" | jq

... Thus, I'd argue that this is not a provider issue per se, but rather a GitLab (API) issue.

---

However, I'm very surprised because even on gitlab.com without a premium license the repository using mirror is actually properly initialized, but it never (even after multiple terraform apply) shows up as Mirroring repository in Settings -> Repository -> Mirror repositories ...).

@igorbrites do you have a premium license for your GitLab repository? (GitLab SaaS license ...)

[timofurrer]

btw. I've created an upstream issue in the meantime to get some help for this: https://gitlab.com/gitlab-org/gitlab/-/issues/364392

[igorbrites]

@timofurrer thanks for answering! Yes, I'm using a premium account, and after applying again, the pull mirroring works as expected.

[psyhomb]

I've encountered exactly the same problem, thanks for reporting the issue @igorbrites.

[psyhomb]

FYI I just finished testing same set of Terraform resources but this time against self-hosted GitLab API (container image: gitlab/gitlab-ee:15.0.1-ee.0) and everything is working on first apply, so it's definitely SaaS GitLab API issue.