Error: Bootstrap run error (failed to push manifests: non-fast-forward update)

[avnes]

In our setup we used a private GitHub repository to bootstrap all our Flux CD installations in a structure like

cluster/<luster-name>/flux-system

Hence we are using the github_repository data provider instead of the github_repository resource.

When bootstrapping Flux CD we get the following error:

Error: Bootstrap run error
│
│   with module.platform_fluxcd.flux_bootstrap_git.this,
│   on ../../_sub/compute/k8s-fluxcd/main.tf line 39, in resource "flux_bootstrap_git" "this":
│   39: resource "flux_bootstrap_git" "this" {
│
│ failed to push manifests: non-fast-forward update: refs/heads/main

It often works fine on the second terraform apply.

This is not surprising since it is a fairly active github repository.

It would be nice if the flux_bootstrap_git resource could have an option to git fetch origin before attempting to write to the repository.

Full code here:

the github provider is passed down from higher up.

terraform {
  required_version = ">= 1.3.0, < 2.0.0"

  required_providers {
    github = {
      source  = "integrations/github"
      version = "~> 5.25.0"
    }
    tls = {
      source  = "hashicorp/tls"
      version = "~> 4.0.4"
    }
    flux = {
      source  = "fluxcd/flux"
      version = "~> 0.25.0"
    }
  }

}

variable "github_owner" {
  type = string
}

variable "repository_name" {
  type = string
}

variable "branch" {
  type    = string
  default = "main"
}

variable "release_tag" {
  type = string
}

variable "cluster_name" {
  type = string
}

variable "kubeconfig_path" {
  type    = string
  default = null
}

locals {
  cluster_target_path = "clusters/${var.cluster_name}"
}

provider "flux" {
  kubernetes = {
    config_path = var.kubeconfig_path
  }
  git = {
    url = "ssh://[email protected]/${data.github_repository.main.full_name}.git"
    ssh = {
      username    = "git"
      private_key = tls_private_key.main.private_key_pem
    }
  }
}

resource "tls_private_key" "main" {
  algorithm   = "ECDSA"
  ecdsa_curve = "P256"
}

data "github_repository" "main" {
  full_name = "${var.github_owner}/${var.repository_name}"
}

data "github_branch" "flux_branch" {
  repository = var.repository_name
  branch     = var.branch
}

resource "github_repository_deploy_key" "main" {
  title      = "fluxcd-${var.cluster_name}"
  repository = data.github_repository.main.name
  key        = tls_private_key.main.public_key_openssh
  read_only  = false
}

resource "flux_bootstrap_git" "this" {
  depends_on = [github_repository_deploy_key.main]
  path       = local.cluster_target_path
  version    = var.release_tag
}

[phillebaba]

This issue is caused by commit occuring during Terraform running. It should be fixed in later releases of the provider by #436.

[Omar-Alkesh]

I am facing the same issue but when creating. The repository already exists, and we are trying to update flux. Already created an issue before finding this one. #479.

[smacintyre]

I'm also facing this same issue, but with gitlab instead of github. Only an issue on the initial creation. Re-planning and re-applying resolve the issue.

[Omar-Alkesh]

Any updates on this issue? It feels like we should use the cli tool instead of using the terraform provider. At least we can add our own retry logic.

---

And @phillebaba, we tried using a provider version with the fix, and it still has issues with pushing to git if something else committed while the bootstrap is running.

[swade1987]

Hello @avnes / @smacintyre ,

I hope you're doing well! I'm the newest contributor to this repository, and I'm currently in the process of issue grooming to ensure that all concerns are addressed promptly and efficiently.

I noticed this issue you reported and wanted to check in with you to see if it's still affecting your work. Your feedback is invaluable to us, and any additional insights or updates you can share would be greatly appreciated to help us understand and solve the problem more effectively.

If this issue has been resolved, could you please share how it was fixed? This information could be incredibly helpful to others in the community facing similar problems. It would also allow us to close this issue with a clear resolution. In case the issue is still open and troubling you, let's work together to find a solution. Your satisfaction and the smooth functioning of our project are our top priorities.

Thank you for your time and contributions to our community. Looking forward to your response!

Best regards,

Steve

---

The issue is that flux expects the repository to be initialised, therefore the fixes are as follows:

• Use the auto_init variable here when creating the Github repository.

• Use the initialize_with_readme variable here when creating your Gitlab project.