terraform-provider-github icon indicating copy to clipboard operation
terraform-provider-github copied to clipboard

Published 20 hours ago • verified publisher icon integrations

Slow performance when managing dozens of repositories

Open mwarkentin opened this issue 3 years ago • 28 comments

Terraform Version

0.12.6

Affected Resource(s)

Please list the resources as a list, for example:

  • github_repository
  • github_branch_protection
  • github_team_repository
  • github_actions_secret

Terraform Configuration Files

Here's our repo module (slightly redacted ****):

terraform {
  required_providers {
    github = ">= 3.1.0"
  }
}

locals {
  # Terraform modules must be named `terraform-<provider>-<module name>`
  # so we can extract the provider easily
  provider = element(split("-", var.repository), 1)
}

data "github_team" "****" {
  slug = "****"
}

data "github_team" "****" {
  slug = "****"
}

resource "github_repository" "main" {
  name        = var.repository
  description = var.description

  visibility = var.visibility

  topics = [
    "terraform",
    "terraform-module",
    "terraform-${local.provider}"
  ]

  has_issues   = var.has_issues
  has_projects = var.has_projects
  has_wiki     = var.has_wiki

  vulnerability_alerts   = true
  delete_branch_on_merge = true

  archived = var.archived

  dynamic "template" {
    for_each = var.fork ? [] : [var.fork]

    content {
      owner      = "waveaccounting"
      repository = "****"
    }
  }
}

resource "github_branch_protection" "main" {
  repository_id = github_repository.main.node_id
  pattern       = github_repository.main.default_branch

  required_status_checks {
    strict = true
    contexts = [
      "Terraform",
      "docs",
    ]
  }

  required_pull_request_reviews {
    dismiss_stale_reviews      = true
    require_code_owner_reviews = true
  }
}

resource "github_team_repository" "****" {
  team_id    = data.github_team.****.id
  repository = github_repository.main.name
  permission = "admin"
}

resource "github_team_repository" "****" {
  team_id    = data.github_team.****.id
  repository = github_repository.main.name
  permission = "admin"
}

resource "github_actions_secret" "secrets" {
  for_each = var.secrets

  repository      = github_repository.main.name
  secret_name     = each.key
  plaintext_value = each.value
}

Actual Behavior

We are managing approximately 90 repositories using this module via Terraform Cloud remote operations (which means we can't disable refresh or change parallelization afaik). I timed a refresh + plan: 9m22s (562s) == 6.2s per repository

Are there any optimizations we can make on our side or in the github provider / API to try to improve this? We're discussing breaking up our repos into smaller workspaces, but that feels like a bit of a hack.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

  1. terraform plan on large numbers of repositories / branch protection configs

Important Factoids

  • Running on Terraform Cloud Remote Operation

References

  • Similar issue to https://github.com/terraform-providers/terraform-provider-github/issues/565, although things weren't particularly fast before the update either

mwarkentin avatar Oct 14 '20 14:10 mwarkentin

this is also true for v3.0.0, and has effectively rendered this provider unsable at v3.x with more than 5 repos while managing files, labels, and permissions of each repo...

reverting back to 2.9.2 until this is resolved

ahmadnassri avatar Oct 17 '20 17:10 ahmadnassri

Thanks for reporting this and providing the example config as well as timing measurements. 🙇

6.2s per repository

We can start with bringing this number down. Not sure how yet, but after some reading for https://github.com/terraform-providers/terraform-provider-github/issues/565, we may be able to collapse a few requests into one with GraphQL 🤔 .

jcudit avatar Oct 19 '20 15:10 jcudit

Took a quick pass at this using the following generated test file:

for i in $(seq 1 100); do cat <<EOF >> main.tf
  resource "github_repository" "repo${i}" {
    name      = "repo${i}"
    auto_init = true
  }
EOF
done

The test was able to refresh 100 repos in 12s:

real	0m12.207s
user	0m1.643s
sys	0m0.584s
details 

$  time terraform refresh   -var "owner=${GITHUB_OWNER}"   -var "github_token=${GITHUB_TOKEN}"
github_repository.repo63: Refreshing state... [id=repo63]
github_repository.repo9: Refreshing state... [id=repo9]
github_repository.repo23: Refreshing state... [id=repo23]
github_repository.repo97: Refreshing state... [id=repo97]
github_repository.repo68: Refreshing state... [id=repo68]
github_repository.repo54: Refreshing state... [id=repo54]
github_repository.repo32: Refreshing state... [id=repo32]
github_repository.repo26: Refreshing state... [id=repo26]
github_repository.repo77: Refreshing state... [id=repo77]
github_repository.repo20: Refreshing state... [id=repo20]
github_repository.repo69: Refreshing state... [id=repo69]
github_repository.repo78: Refreshing state... [id=repo78]
github_repository.repo6: Refreshing state... [id=repo6]
github_repository.repo4: Refreshing state... [id=repo4]
github_repository.repo31: Refreshing state... [id=repo31]
github_repository.repo64: Refreshing state... [id=repo64]
github_repository.repo72: Refreshing state... [id=repo72]
github_repository.repo38: Refreshing state... [id=repo38]
github_repository.repo15: Refreshing state... [id=repo15]
github_repository.repo43: Refreshing state... [id=repo43]
github_repository.repo44: Refreshing state... [id=repo44]
github_repository.repo18: Refreshing state... [id=repo18]
github_repository.repo96: Refreshing state... [id=repo96]
github_repository.repo99: Refreshing state... [id=repo99]
github_repository.repo81: Refreshing state... [id=repo81]
github_repository.repo61: Refreshing state... [id=repo61]
github_repository.repo22: Refreshing state... [id=repo22]
github_repository.repo17: Refreshing state... [id=repo17]
github_repository.repo7: Refreshing state... [id=repo7]
github_repository.repo56: Refreshing state... [id=repo56]
github_repository.repo86: Refreshing state... [id=repo86]
github_repository.repo2: Refreshing state... [id=repo2]
github_repository.repo28: Refreshing state... [id=repo28]
github_repository.repo82: Refreshing state... [id=repo82]
github_repository.repo39: Refreshing state... [id=repo39]
github_repository.repo73: Refreshing state... [id=repo73]
github_repository.repo92: Refreshing state... [id=repo92]
github_repository.repo66: Refreshing state... [id=repo66]
github_repository.repo11: Refreshing state... [id=repo11]
github_repository.repo89: Refreshing state... [id=repo89]
github_repository.repo42: Refreshing state... [id=repo42]
github_repository.repo41: Refreshing state... [id=repo41]
github_repository.repo16: Refreshing state... [id=repo16]
github_repository.repo37: Refreshing state... [id=repo37]
github_repository.repo79: Refreshing state... [id=repo79]
github_repository.repo85: Refreshing state... [id=repo85]
github_repository.repo12: Refreshing state... [id=repo12]
github_repository.repo21: Refreshing state... [id=repo21]
github_repository.repo58: Refreshing state... [id=repo58]
github_repository.repo8: Refreshing state... [id=repo8]
github_repository.repo83: Refreshing state... [id=repo83]
github_repository.repo19: Refreshing state... [id=repo19]
github_repository.repo76: Refreshing state... [id=repo76]
github_repository.repo71: Refreshing state... [id=repo71]
github_repository.repo52: Refreshing state... [id=repo52]
github_repository.repo55: Refreshing state... [id=repo55]
github_repository.repo75: Refreshing state... [id=repo75]
github_repository.repo10: Refreshing state... [id=repo10]
github_repository.repo33: Refreshing state... [id=repo33]
github_repository.repo53: Refreshing state... [id=repo53]
github_repository.repo36: Refreshing state... [id=repo36]
github_repository.repo62: Refreshing state... [id=repo62]
github_repository.repo49: Refreshing state... [id=repo49]
github_repository.repo91: Refreshing state... [id=repo91]
github_repository.repo5: Refreshing state... [id=repo5]
github_repository.repo46: Refreshing state... [id=repo46]
github_repository.repo24: Refreshing state... [id=repo24]
github_repository.repo29: Refreshing state... [id=repo29]
github_repository.repo74: Refreshing state... [id=repo74]
github_repository.repo57: Refreshing state... [id=repo57]
github_repository.repo1: Refreshing state... [id=repo1]
github_repository.repo45: Refreshing state... [id=repo45]
github_repository.repo14: Refreshing state... [id=repo14]
github_repository.repo67: Refreshing state... [id=repo67]
github_repository.repo40: Refreshing state... [id=repo40]
github_repository.repo70: Refreshing state... [id=repo70]
github_repository.repo94: Refreshing state... [id=repo94]
github_repository.repo100: Refreshing state... [id=repo100]
github_repository.repo93: Refreshing state... [id=repo93]
github_repository.repo35: Refreshing state... [id=repo35]
github_repository.repo51: Refreshing state... [id=repo51]
github_repository.repo59: Refreshing state... [id=repo59]
github_repository.repo84: Refreshing state... [id=repo84]
github_repository.repo25: Refreshing state... [id=repo25]
github_repository.repo50: Refreshing state... [id=repo50]
github_repository.repo90: Refreshing state... [id=repo90]
github_repository.repo95: Refreshing state... [id=repo95]
github_repository.repo34: Refreshing state... [id=repo34]
github_repository.repo88: Refreshing state... [id=repo88]
github_repository.repo65: Refreshing state... [id=repo65]
github_repository.repo48: Refreshing state... [id=repo48]
github_repository.repo3: Refreshing state... [id=repo3]
github_repository.repo98: Refreshing state... [id=repo98]
github_repository.repo87: Refreshing state... [id=repo87]
github_repository.repo13: Refreshing state... [id=repo13]
github_repository.repo30: Refreshing state... [id=repo30]
github_repository.repo80: Refreshing state... [id=repo80]
github_repository.repo47: Refreshing state... [id=repo47]
github_repository.repo60: Refreshing state... [id=repo60]
github_repository.repo27: Refreshing state... [id=repo27]

real	0m12.207s
user	0m1.643s
sys	0m0.584s

I don't doubt things are slow for your setup, however I'd like more information to better reproduce this.
@mwarkentin (or anyone else following) can I get counts of each of the resources?

terraform state list | cut -d. -f1 | uniq -c or similar would help!

jcudit avatar Oct 20 '20 00:10 jcudit

@jcudit github_branch_protection and github_repository_file is where I've seen the massive speed and rate limit issues

ahmadnassri avatar Oct 20 '20 01:10 ahmadnassri

Thanks for confirming. Adding github_branch_protection to the test above reveals this poor performance.

for i in $(seq 1 100); do cat <<EOF >> main.tf
  resource "github_repository" "repo${i}" {
    name      = "repo${i}"
    auto_init = true
  }
  resource "github_branch_protection" "repo${i}" {
    repository_id = github_repository.repo${i}.node_id
    pattern       = github_repository.repo${i}.default_branch
    required_status_checks {
      strict = true
      contexts = [
        "Terraform",
        "docs",
      ]
    }
    required_pull_request_reviews {
      dismiss_stale_reviews      = true
      require_code_owner_reviews = true
    }
  }
EOF
done

Will focus efforts there and will track github_repository_file in https://github.com/terraform-providers/terraform-provider-github/issues/568 🙇

jcudit avatar Oct 20 '20 01:10 jcudit

@jcudit if it's still helpful:

❯ terraform state list | cut -d. -f3 | sort | uniq -c
 194 data
  96 github_actions_secret
  96 github_branch_protection
  96 github_repository
 194 github_team_repository

mwarkentin avatar Oct 20 '20 01:10 mwarkentin

Performed a round of measurements that compared v3.1.0 and v2.9.2 (GraphQL vs. REST implementations). Some observations below:

  • For a single repository, performance was similar between versions (numbers below)
  • For 20 repositories, the REST implementation took ~10 seconds to refresh.
  • For 20 repositories, the GraphQL implementation took ~25 seconds to refresh.
  • For 100 repositories, the REST implementation took ~40 seconds to refresh.
  • For 100 repositories, the GraphQL implementation took ~130 seconds to refresh.
  • The GraphQL implementation began spacing requests to 1/second after an initial burst of sub-second spacing
v3.1.0 

$ time terraform refresh   -var "owner=${GITHUB_OWNER}"   -var "github_token=${GITHUB_TOKEN}"
github_repository.repo: Refreshing state... [id=repo]
github_branch_protection.repo: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMTU4MTM=]

real	0m1.210s
user	0m0.747s
sys	0m0.417s

$ time terraform refresh   -var "owner=${GITHUB_OWNER}"   -var "github_token=${GITHUB_TOKEN}"
github_repository.repo11: Refreshing state... [id=repo11]
github_repository.repo9: Refreshing state... [id=repo9]
github_repository.repo12: Refreshing state... [id=repo12]
github_repository.repo13: Refreshing state... [id=repo13]
github_repository.repo8: Refreshing state... [id=repo8]
github_repository.repo1: Refreshing state... [id=repo1]
github_repository.repo6: Refreshing state... [id=repo6]
github_repository.repo17: Refreshing state... [id=repo17]
github_repository.repo19: Refreshing state... [id=repo19]
github_repository.repo7: Refreshing state... [id=repo7]
github_repository.repo3: Refreshing state... [id=repo3]
github_repository.repo16: Refreshing state... [id=repo16]
github_repository.repo4: Refreshing state... [id=repo4]
github_repository.repo2: Refreshing state... [id=repo2]
github_repository.repo15: Refreshing state... [id=repo15]
github_repository.repo14: Refreshing state... [id=repo14]
github_repository.repo18: Refreshing state... [id=repo18]
github_repository.repo10: Refreshing state... [id=repo10]
github_repository.repo5: Refreshing state... [id=repo5]
github_repository.repo20: Refreshing state... [id=repo20]
github_branch_protection.repo9: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTA=]
github_branch_protection.repo12: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyNzI=]
github_branch_protection.repo8: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUzMDY=]
github_branch_protection.repo11: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTY=]
github_branch_protection.repo13: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODk=]
github_branch_protection.repo1: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTc=]
github_branch_protection.repo17: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODY=]
github_branch_protection.repo6: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTg=]
github_branch_protection.repo19: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUzMDM=]
github_branch_protection.repo7: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODQ=]
github_branch_protection.repo3: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTU=]
github_branch_protection.repo16: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUzMDE=]
github_branch_protection.repo4: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODM=]
github_branch_protection.repo2: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODE=]
github_branch_protection.repo15: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODU=]
github_branch_protection.repo14: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTk=]
github_branch_protection.repo18: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTQ=]
github_branch_protection.repo10: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODg=]
github_branch_protection.repo5: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyNzQ=]
github_branch_protection.repo20: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUzMDI=]

real	0m25.289s
user	0m1.162s
sys	0m0.625s

$  time terraform refresh   -var "owner=${GITHUB_OWNER}"   -var "github_token=${GITHUB_TOKEN}"
github_repository.repo53: Refreshing state... [id=repo53]
github_repository.repo26: Refreshing state... [id=repo26]
github_repository.repo63: Refreshing state... [id=repo63]
github_repository.repo15: Refreshing state... [id=repo15]
github_repository.repo9: Refreshing state... [id=repo9]
github_repository.repo24: Refreshing state... [id=repo24]
github_repository.repo25: Refreshing state... [id=repo25]
github_repository.repo79: Refreshing state... [id=repo79]
github_repository.repo17: Refreshing state... [id=repo17]
github_repository.repo82: Refreshing state... [id=repo82]
github_repository.repo60: Refreshing state... [id=repo60]
github_repository.repo4: Refreshing state... [id=repo4]
github_repository.repo68: Refreshing state... [id=repo68]
github_repository.repo23: Refreshing state... [id=repo23]
github_repository.repo74: Refreshing state... [id=repo74]
github_repository.repo95: Refreshing state... [id=repo95]
github_repository.repo59: Refreshing state... [id=repo59]
github_repository.repo48: Refreshing state... [id=repo48]
github_repository.repo61: Refreshing state... [id=repo61]
github_repository.repo96: Refreshing state... [id=repo96]
github_repository.repo83: Refreshing state... [id=repo83]
github_repository.repo90: Refreshing state... [id=repo90]
github_repository.repo47: Refreshing state... [id=repo47]
github_repository.repo52: Refreshing state... [id=repo52]
github_repository.repo88: Refreshing state... [id=repo88]
github_repository.repo19: Refreshing state... [id=repo19]
github_repository.repo78: Refreshing state... [id=repo78]
github_repository.repo39: Refreshing state... [id=repo39]
github_repository.repo69: Refreshing state... [id=repo69]
github_repository.repo37: Refreshing state... [id=repo37]
github_repository.repo31: Refreshing state... [id=repo31]
github_repository.repo73: Refreshing state... [id=repo73]
github_repository.repo57: Refreshing state... [id=repo57]
github_repository.repo71: Refreshing state... [id=repo71]
github_repository.repo99: Refreshing state... [id=repo99]
github_repository.repo44: Refreshing state... [id=repo44]
github_repository.repo65: Refreshing state... [id=repo65]
github_repository.repo64: Refreshing state... [id=repo64]
github_repository.repo72: Refreshing state... [id=repo72]
github_repository.repo16: Refreshing state... [id=repo16]
github_repository.repo18: Refreshing state... [id=repo18]
github_repository.repo80: Refreshing state... [id=repo80]
github_repository.repo49: Refreshing state... [id=repo49]
github_repository.repo66: Refreshing state... [id=repo66]
github_repository.repo75: Refreshing state... [id=repo75]
github_repository.repo81: Refreshing state... [id=repo81]
github_repository.repo8: Refreshing state... [id=repo8]
github_repository.repo34: Refreshing state... [id=repo34]
github_repository.repo70: Refreshing state... [id=repo70]
github_repository.repo42: Refreshing state... [id=repo42]
github_repository.repo86: Refreshing state... [id=repo86]
github_repository.repo14: Refreshing state... [id=repo14]
github_repository.repo12: Refreshing state... [id=repo12]
github_repository.repo6: Refreshing state... [id=repo6]
github_repository.repo51: Refreshing state... [id=repo51]
github_repository.repo62: Refreshing state... [id=repo62]
github_repository.repo33: Refreshing state... [id=repo33]
github_repository.repo84: Refreshing state... [id=repo84]
github_repository.repo38: Refreshing state... [id=repo38]
github_repository.repo36: Refreshing state... [id=repo36]
github_repository.repo13: Refreshing state... [id=repo13]
github_repository.repo32: Refreshing state... [id=repo32]
github_repository.repo98: Refreshing state... [id=repo98]
github_repository.repo58: Refreshing state... [id=repo58]
github_repository.repo40: Refreshing state... [id=repo40]
github_repository.repo56: Refreshing state... [id=repo56]
github_repository.repo41: Refreshing state... [id=repo41]
github_repository.repo92: Refreshing state... [id=repo92]
github_repository.repo1: Refreshing state... [id=repo1]
github_repository.repo30: Refreshing state... [id=repo30]
github_repository.repo97: Refreshing state... [id=repo97]
github_repository.repo87: Refreshing state... [id=repo87]
github_repository.repo28: Refreshing state... [id=repo28]
github_repository.repo91: Refreshing state... [id=repo91]
github_repository.repo55: Refreshing state... [id=repo55]
github_repository.repo29: Refreshing state... [id=repo29]
github_repository.repo93: Refreshing state... [id=repo93]
github_repository.repo7: Refreshing state... [id=repo7]
github_repository.repo2: Refreshing state... [id=repo2]
github_repository.repo54: Refreshing state... [id=repo54]
github_repository.repo85: Refreshing state... [id=repo85]
github_repository.repo11: Refreshing state... [id=repo11]
github_repository.repo22: Refreshing state... [id=repo22]
github_repository.repo27: Refreshing state... [id=repo27]
github_repository.repo100: Refreshing state... [id=repo100]
github_repository.repo5: Refreshing state... [id=repo5]
github_repository.repo76: Refreshing state... [id=repo76]
github_repository.repo67: Refreshing state... [id=repo67]
github_repository.repo89: Refreshing state... [id=repo89]
github_repository.repo21: Refreshing state... [id=repo21]
github_repository.repo35: Refreshing state... [id=repo35]
github_repository.repo45: Refreshing state... [id=repo45]
github_repository.repo3: Refreshing state... [id=repo3]
github_repository.repo50: Refreshing state... [id=repo50]
github_repository.repo46: Refreshing state... [id=repo46]
github_repository.repo77: Refreshing state... [id=repo77]
github_repository.repo43: Refreshing state... [id=repo43]
github_repository.repo20: Refreshing state... [id=repo20]
github_repository.repo94: Refreshing state... [id=repo94]
github_repository.repo10: Refreshing state... [id=repo10]
github_branch_protection.repo26: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MjE=]
github_branch_protection.repo63: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU4MTM=]
github_branch_protection.repo53: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3MjQ=]
github_branch_protection.repo15: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODU=]
github_branch_protection.repo9: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTA=]
github_branch_protection.repo24: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2Mzk=]
github_branch_protection.repo25: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3OTM=]
github_branch_protection.repo17: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODY=]
github_branch_protection.repo79: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NTM=]
github_branch_protection.repo82: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3Njc=]
github_branch_protection.repo60: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NTc=]
github_branch_protection.repo4: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODM=]
github_branch_protection.repo68: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU4MTE=]
github_branch_protection.repo23: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3MzU=]
github_branch_protection.repo74: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NTQ=]
github_branch_protection.repo95: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3MDk=]
github_branch_protection.repo59: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MzA=]
github_branch_protection.repo48: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MjU=]
github_branch_protection.repo61: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3NjE=]
github_branch_protection.repo96: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2Mjc=]
github_branch_protection.repo83: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2Nzc=]
github_branch_protection.repo90: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MTQ=]
github_branch_protection.repo47: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NDQ=]
github_branch_protection.repo52: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3NzI=]
github_branch_protection.repo88: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MjY=]
github_branch_protection.repo19: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUzMDM=]
github_branch_protection.repo78: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2Njk=]
github_branch_protection.repo39: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3NTY=]
github_branch_protection.repo69: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MTY=]
github_branch_protection.repo37: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3NzQ=]
github_branch_protection.repo31: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU4MDY=]
github_branch_protection.repo73: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3Nzc=]
github_branch_protection.repo57: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MjM=]
github_branch_protection.repo71: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NTY=]
github_branch_protection.repo99: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MDY=]
github_branch_protection.repo44: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MTM=]
github_branch_protection.repo65: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3Mjc=]
github_branch_protection.repo64: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NDA=]
github_branch_protection.repo72: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3OTg=]
github_branch_protection.repo16: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUzMDE=]
github_branch_protection.repo18: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTQ=]
github_branch_protection.repo80: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2Nzk=]
github_branch_protection.repo49: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3OTY=]
github_branch_protection.repo66: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2Mjk=]
github_branch_protection.repo75: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NDk=]
github_branch_protection.repo81: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MDk=]
github_branch_protection.repo8: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUzMDY=]
github_branch_protection.repo34: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MzY=]
github_branch_protection.repo70: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU4MDg=]
github_branch_protection.repo42: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NDE=]
github_branch_protection.repo86: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MDU=]
github_branch_protection.repo14: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTk=]
github_branch_protection.repo12: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyNzI=]
github_branch_protection.repo6: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTg=]
github_branch_protection.repo51: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3MTM=]
github_branch_protection.repo62: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU4MDE=]
github_branch_protection.repo33: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MTA=]
github_branch_protection.repo84: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NTE=]
github_branch_protection.repo38: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NzU=]
github_branch_protection.repo36: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2Mzc=]
github_branch_protection.repo13: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODk=]
github_branch_protection.repo32: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3MzI=]
github_branch_protection.repo98: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3MTg=]
github_branch_protection.repo58: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2ODc=]
github_branch_protection.repo40: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3NjQ=]
github_branch_protection.repo56: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MTI=]
github_branch_protection.repo41: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NDI=]
github_branch_protection.repo92: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3MzA=]
github_branch_protection.repo1: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTc=]
github_branch_protection.repo30: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NTA=]
github_branch_protection.repo97: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MDQ=]
github_branch_protection.repo87: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3Njk=]
github_branch_protection.repo28: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2ODk=]
github_branch_protection.repo91: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3NTk=]
github_branch_protection.repo55: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MjI=]
github_branch_protection.repo29: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3MTY=]
github_branch_protection.repo93: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MTk=]
github_branch_protection.repo7: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODQ=]
github_branch_protection.repo2: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODE=]
github_branch_protection.repo54: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NDg=]
github_branch_protection.repo85: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3OTA=]
github_branch_protection.repo11: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTY=]
github_branch_protection.repo22: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NTI=]
github_branch_protection.repo27: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2ODQ=]
github_branch_protection.repo100: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NzI=]
github_branch_protection.repo5: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyNzQ=]
github_branch_protection.repo76: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3NTE=]
github_branch_protection.repo67: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MzU=]
github_branch_protection.repo89: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MTc=]
github_branch_protection.repo21: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2ODI=]
github_branch_protection.repo35: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NDU=]
github_branch_protection.repo45: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MzQ=]
github_branch_protection.repo3: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyOTU=]
github_branch_protection.repo50: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2Mjg=]
github_branch_protection.repo46: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3MjE=]
github_branch_protection.repo77: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2OTQ=]
github_branch_protection.repo43: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2NDc=]
github_branch_protection.repo20: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUzMDI=]
github_branch_protection.repo94: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU2MjQ=]
github_branch_protection.repo10: Refreshing state... [id=MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjUyODg=]

real	2m10.649s
user	0m2.623s
sys	0m1.333s
v2.9.2 

$ time terraform refresh  -var "owner=${GITHUB_OWNER}"  -var "github_token=${GITHUB_TOKEN}"
github_repository.repo: Refreshing state... [id=repo]
github_branch_protection.repo: Refreshing state... [id=repo:main]

real	0m1.356s
user	0m0.759s
sys	0m0.416s

time terraform refresh   -var "owner=${GITHUB_OWNER}"   -var "github_token=${GITHUB_TOKEN}"
github_repository.repo15: Refreshing state... [id=repo15]
github_repository.repo6: Refreshing state... [id=repo6]
github_repository.repo17: Refreshing state... [id=repo17]
github_repository.repo13: Refreshing state... [id=repo13]
github_repository.repo10: Refreshing state... [id=repo10]
github_repository.repo8: Refreshing state... [id=repo8]
github_repository.repo12: Refreshing state... [id=repo12]
github_repository.repo19: Refreshing state... [id=repo19]
github_repository.repo5: Refreshing state... [id=repo5]
github_repository.repo4: Refreshing state... [id=repo4]
github_repository.repo18: Refreshing state... [id=repo18]
github_repository.repo16: Refreshing state... [id=repo16]
github_repository.repo20: Refreshing state... [id=repo20]
github_repository.repo11: Refreshing state... [id=repo11]
github_repository.repo1: Refreshing state... [id=repo1]
github_repository.repo3: Refreshing state... [id=repo3]
github_repository.repo2: Refreshing state... [id=repo2]
github_repository.repo9: Refreshing state... [id=repo9]
github_repository.repo14: Refreshing state... [id=repo14]
github_repository.repo7: Refreshing state... [id=repo7]
github_branch_protection.repo6: Refreshing state... [id=repo6:main]
github_branch_protection.repo15: Refreshing state... [id=repo15:main]
github_branch_protection.repo17: Refreshing state... [id=repo17:main]
github_branch_protection.repo10: Refreshing state... [id=repo10:main]
github_branch_protection.repo13: Refreshing state... [id=repo13:main]
github_branch_protection.repo8: Refreshing state... [id=repo8:main]
github_branch_protection.repo12: Refreshing state... [id=repo12:main]
github_branch_protection.repo19: Refreshing state... [id=repo19:main]
github_branch_protection.repo5: Refreshing state... [id=repo5:main]
github_branch_protection.repo4: Refreshing state... [id=repo4:main]
github_branch_protection.repo18: Refreshing state... [id=repo18:main]
github_branch_protection.repo16: Refreshing state... [id=repo16:main]
github_branch_protection.repo20: Refreshing state... [id=repo20:main]
github_branch_protection.repo11: Refreshing state... [id=repo11:main]
github_branch_protection.repo1: Refreshing state... [id=repo1:main]
github_branch_protection.repo3: Refreshing state... [id=repo3:main]
github_branch_protection.repo2: Refreshing state... [id=repo2:main]
github_branch_protection.repo9: Refreshing state... [id=repo9:main]
github_branch_protection.repo14: Refreshing state... [id=repo14:main]
github_branch_protection.repo7: Refreshing state... [id=repo7:main]

real	0m8.650s
user	0m1.157s
sys	0m0.605s

$  time terraform refresh   -var "owner=${GITHUB_OWNER}"   -var "github_token=${GITHUB_TOKEN}"
github_repository.repo20: Refreshing state... [id=repo20]
github_repository.repo45: Refreshing state... [id=repo45]
github_repository.repo80: Refreshing state... [id=repo80]
github_repository.repo54: Refreshing state... [id=repo54]
github_repository.repo68: Refreshing state... [id=repo68]
github_repository.repo51: Refreshing state... [id=repo51]
github_repository.repo26: Refreshing state... [id=repo26]
github_repository.repo91: Refreshing state... [id=repo91]
github_repository.repo4: Refreshing state... [id=repo4]
github_repository.repo76: Refreshing state... [id=repo76]
github_repository.repo78: Refreshing state... [id=repo78]
github_repository.repo65: Refreshing state... [id=repo65]
github_repository.repo15: Refreshing state... [id=repo15]
github_repository.repo47: Refreshing state... [id=repo47]
github_repository.repo25: Refreshing state... [id=repo25]
github_repository.repo73: Refreshing state... [id=repo73]
github_repository.repo37: Refreshing state... [id=repo37]
github_repository.repo93: Refreshing state... [id=repo93]
github_repository.repo86: Refreshing state... [id=repo86]
github_repository.repo81: Refreshing state... [id=repo81]
github_repository.repo43: Refreshing state... [id=repo43]
github_repository.repo17: Refreshing state... [id=repo17]
github_repository.repo100: Refreshing state... [id=repo100]
github_repository.repo83: Refreshing state... [id=repo83]
github_repository.repo66: Refreshing state... [id=repo66]
github_repository.repo85: Refreshing state... [id=repo85]
github_repository.repo94: Refreshing state... [id=repo94]
github_repository.repo33: Refreshing state... [id=repo33]
github_repository.repo62: Refreshing state... [id=repo62]
github_repository.repo9: Refreshing state... [id=repo9]
github_repository.repo82: Refreshing state... [id=repo82]
github_repository.repo74: Refreshing state... [id=repo74]
github_repository.repo13: Refreshing state... [id=repo13]
github_repository.repo41: Refreshing state... [id=repo41]
github_repository.repo27: Refreshing state... [id=repo27]
github_repository.repo77: Refreshing state... [id=repo77]
github_repository.repo87: Refreshing state... [id=repo87]
github_repository.repo96: Refreshing state... [id=repo96]
github_repository.repo8: Refreshing state... [id=repo8]
github_repository.repo40: Refreshing state... [id=repo40]
github_repository.repo52: Refreshing state... [id=repo52]
github_repository.repo97: Refreshing state... [id=repo97]
github_repository.repo48: Refreshing state... [id=repo48]
github_repository.repo32: Refreshing state... [id=repo32]
github_repository.repo58: Refreshing state... [id=repo58]
github_repository.repo69: Refreshing state... [id=repo69]
github_repository.repo24: Refreshing state... [id=repo24]
github_repository.repo35: Refreshing state... [id=repo35]
github_repository.repo7: Refreshing state... [id=repo7]
github_repository.repo42: Refreshing state... [id=repo42]
github_repository.repo75: Refreshing state... [id=repo75]
github_repository.repo60: Refreshing state... [id=repo60]
github_repository.repo30: Refreshing state... [id=repo30]
github_repository.repo99: Refreshing state... [id=repo99]
github_repository.repo18: Refreshing state... [id=repo18]
github_repository.repo79: Refreshing state... [id=repo79]
github_repository.repo53: Refreshing state... [id=repo53]
github_repository.repo22: Refreshing state... [id=repo22]
github_repository.repo59: Refreshing state... [id=repo59]
github_repository.repo55: Refreshing state... [id=repo55]
github_repository.repo95: Refreshing state... [id=repo95]
github_repository.repo88: Refreshing state... [id=repo88]
github_repository.repo89: Refreshing state... [id=repo89]
github_repository.repo12: Refreshing state... [id=repo12]
github_repository.repo56: Refreshing state... [id=repo56]
github_repository.repo84: Refreshing state... [id=repo84]
github_repository.repo61: Refreshing state... [id=repo61]
github_repository.repo3: Refreshing state... [id=repo3]
github_repository.repo90: Refreshing state... [id=repo90]
github_repository.repo21: Refreshing state... [id=repo21]
github_repository.repo6: Refreshing state... [id=repo6]
github_repository.repo46: Refreshing state... [id=repo46]
github_repository.repo63: Refreshing state... [id=repo63]
github_repository.repo71: Refreshing state... [id=repo71]
github_repository.repo1: Refreshing state... [id=repo1]
github_repository.repo72: Refreshing state... [id=repo72]
github_repository.repo98: Refreshing state... [id=repo98]
github_repository.repo14: Refreshing state... [id=repo14]
github_repository.repo70: Refreshing state... [id=repo70]
github_repository.repo10: Refreshing state... [id=repo10]
github_repository.repo67: Refreshing state... [id=repo67]
github_repository.repo19: Refreshing state... [id=repo19]
github_repository.repo28: Refreshing state... [id=repo28]
github_repository.repo29: Refreshing state... [id=repo29]
github_repository.repo5: Refreshing state... [id=repo5]
github_repository.repo36: Refreshing state... [id=repo36]
github_repository.repo39: Refreshing state... [id=repo39]
github_repository.repo44: Refreshing state... [id=repo44]
github_repository.repo57: Refreshing state... [id=repo57]
github_repository.repo2: Refreshing state... [id=repo2]
github_repository.repo38: Refreshing state... [id=repo38]
github_repository.repo34: Refreshing state... [id=repo34]
github_repository.repo49: Refreshing state... [id=repo49]
github_repository.repo23: Refreshing state... [id=repo23]
github_repository.repo11: Refreshing state... [id=repo11]
github_repository.repo64: Refreshing state... [id=repo64]
github_repository.repo31: Refreshing state... [id=repo31]
github_repository.repo16: Refreshing state... [id=repo16]
github_repository.repo92: Refreshing state... [id=repo92]
github_repository.repo50: Refreshing state... [id=repo50]
github_branch_protection.repo45: Refreshing state... [id=repo45:main]
github_branch_protection.repo20: Refreshing state... [id=repo20:main]
github_branch_protection.repo80: Refreshing state... [id=repo80:main]
github_branch_protection.repo54: Refreshing state... [id=repo54:main]
github_branch_protection.repo68: Refreshing state... [id=repo68:main]
github_branch_protection.repo51: Refreshing state... [id=repo51:main]
github_branch_protection.repo26: Refreshing state... [id=repo26:main]
github_branch_protection.repo91: Refreshing state... [id=repo91:main]
github_branch_protection.repo4: Refreshing state... [id=repo4:main]
github_branch_protection.repo76: Refreshing state... [id=repo76:main]
github_branch_protection.repo78: Refreshing state... [id=repo78:main]
github_branch_protection.repo65: Refreshing state... [id=repo65:main]
github_branch_protection.repo15: Refreshing state... [id=repo15:main]
github_branch_protection.repo47: Refreshing state... [id=repo47:main]
github_branch_protection.repo25: Refreshing state... [id=repo25:main]
github_branch_protection.repo73: Refreshing state... [id=repo73:main]
github_branch_protection.repo37: Refreshing state... [id=repo37:main]
github_branch_protection.repo93: Refreshing state... [id=repo93:main]
github_branch_protection.repo86: Refreshing state... [id=repo86:main]
github_branch_protection.repo81: Refreshing state... [id=repo81:main]
github_branch_protection.repo43: Refreshing state... [id=repo43:main]
github_branch_protection.repo17: Refreshing state... [id=repo17:main]
github_branch_protection.repo100: Refreshing state... [id=repo100:main]
github_branch_protection.repo83: Refreshing state... [id=repo83:main]
github_branch_protection.repo66: Refreshing state... [id=repo66:main]
github_branch_protection.repo85: Refreshing state... [id=repo85:main]
github_branch_protection.repo94: Refreshing state... [id=repo94:main]
github_branch_protection.repo33: Refreshing state... [id=repo33:main]
github_branch_protection.repo62: Refreshing state... [id=repo62:main]
github_branch_protection.repo9: Refreshing state... [id=repo9:main]
github_branch_protection.repo82: Refreshing state... [id=repo82:main]
github_branch_protection.repo74: Refreshing state... [id=repo74:main]
github_branch_protection.repo13: Refreshing state... [id=repo13:main]
github_branch_protection.repo41: Refreshing state... [id=repo41:main]
github_branch_protection.repo27: Refreshing state... [id=repo27:main]
github_branch_protection.repo77: Refreshing state... [id=repo77:main]
github_branch_protection.repo87: Refreshing state... [id=repo87:main]
github_branch_protection.repo96: Refreshing state... [id=repo96:main]
github_branch_protection.repo8: Refreshing state... [id=repo8:main]
github_branch_protection.repo40: Refreshing state... [id=repo40:main]
github_branch_protection.repo52: Refreshing state... [id=repo52:main]
github_branch_protection.repo97: Refreshing state... [id=repo97:main]
github_branch_protection.repo48: Refreshing state... [id=repo48:main]
github_branch_protection.repo32: Refreshing state... [id=repo32:main]
github_branch_protection.repo58: Refreshing state... [id=repo58:main]
github_branch_protection.repo69: Refreshing state... [id=repo69:main]
github_branch_protection.repo24: Refreshing state... [id=repo24:main]
github_branch_protection.repo35: Refreshing state... [id=repo35:main]
github_branch_protection.repo7: Refreshing state... [id=repo7:main]
github_branch_protection.repo42: Refreshing state... [id=repo42:main]
github_branch_protection.repo75: Refreshing state... [id=repo75:main]
github_branch_protection.repo60: Refreshing state... [id=repo60:main]
github_branch_protection.repo30: Refreshing state... [id=repo30:main]
github_branch_protection.repo99: Refreshing state... [id=repo99:main]
github_branch_protection.repo18: Refreshing state... [id=repo18:main]
github_branch_protection.repo79: Refreshing state... [id=repo79:main]
github_branch_protection.repo53: Refreshing state... [id=repo53:main]
github_branch_protection.repo22: Refreshing state... [id=repo22:main]
github_branch_protection.repo59: Refreshing state... [id=repo59:main]
github_branch_protection.repo55: Refreshing state... [id=repo55:main]
github_branch_protection.repo95: Refreshing state... [id=repo95:main]
github_branch_protection.repo88: Refreshing state... [id=repo88:main]
github_branch_protection.repo89: Refreshing state... [id=repo89:main]
github_branch_protection.repo12: Refreshing state... [id=repo12:main]
github_branch_protection.repo56: Refreshing state... [id=repo56:main]
github_branch_protection.repo84: Refreshing state... [id=repo84:main]
github_branch_protection.repo61: Refreshing state... [id=repo61:main]
github_branch_protection.repo3: Refreshing state... [id=repo3:main]
github_branch_protection.repo90: Refreshing state... [id=repo90:main]
github_branch_protection.repo21: Refreshing state... [id=repo21:main]
github_branch_protection.repo6: Refreshing state... [id=repo6:main]
github_branch_protection.repo46: Refreshing state... [id=repo46:main]
github_branch_protection.repo63: Refreshing state... [id=repo63:main]
github_branch_protection.repo71: Refreshing state... [id=repo71:main]
github_branch_protection.repo1: Refreshing state... [id=repo1:main]
github_branch_protection.repo72: Refreshing state... [id=repo72:main]
github_branch_protection.repo98: Refreshing state... [id=repo98:main]
github_branch_protection.repo14: Refreshing state... [id=repo14:main]
github_branch_protection.repo70: Refreshing state... [id=repo70:main]
github_branch_protection.repo10: Refreshing state... [id=repo10:main]
github_branch_protection.repo67: Refreshing state... [id=repo67:main]
github_branch_protection.repo19: Refreshing state... [id=repo19:main]
github_branch_protection.repo28: Refreshing state... [id=repo28:main]
github_branch_protection.repo29: Refreshing state... [id=repo29:main]
github_branch_protection.repo5: Refreshing state... [id=repo5:main]
github_branch_protection.repo36: Refreshing state... [id=repo36:main]
github_branch_protection.repo39: Refreshing state... [id=repo39:main]
github_branch_protection.repo44: Refreshing state... [id=repo44:main]
github_branch_protection.repo57: Refreshing state... [id=repo57:main]
github_branch_protection.repo2: Refreshing state... [id=repo2:main]
github_branch_protection.repo38: Refreshing state... [id=repo38:main]
github_branch_protection.repo34: Refreshing state... [id=repo34:main]
github_branch_protection.repo49: Refreshing state... [id=repo49:main]
github_branch_protection.repo23: Refreshing state... [id=repo23:main]
github_branch_protection.repo11: Refreshing state... [id=repo11:main]
github_branch_protection.repo64: Refreshing state... [id=repo64:main]
github_branch_protection.repo31: Refreshing state... [id=repo31:main]
github_branch_protection.repo16: Refreshing state... [id=repo16:main]
github_branch_protection.repo92: Refreshing state... [id=repo92:main]
github_branch_protection.repo50: Refreshing state... [id=repo50:main]

real	0m39.843s
user	0m2.694s
sys	0m1.368s

jcudit avatar Oct 22 '20 19:10 jcudit

Given the results above, here are some recommendations:

  • Refine the GraphQL query to be less costly (current shown below)
  • Revert the github_branch_protection back to the REST implementation, but handle unsupported configurations with the GraphQL implementation
  • Continue as-is and have users maintain smaller state files allowing higher parallelization
  • Revert the github_branch_protection back to the REST implementation, forfeiting the features available via the GraphQL implementation

I will attempt to get the first option into the upcoming bugfix release.

The second option is larger than I have time for but welcome PRs from contributors.

The last two options will need feedback / sign-off from the community. Interested to hear more opinions here if the first two options do not play out well.

current query 

{
  "query": "query($id:ID!){node(id: $id){... on BranchProtectionRule{repository{id,name},pushAllowances(first: 100){nodes{actor{... on Team{id,name},... on User{id,name}}}},reviewDismissalAllowances(first: 100){nodes{actor{... on Team{id,name},... on User{id,name}}}},dismissesStaleReviews,id,isAdminEnforced,pattern,requiredApprovingReviewCount,requiredStatusCheckContexts,requiresApprovingReviews,requiresCodeOwnerReviews,requiresCommitSignatures,requiresStatusChecks,requiresStrictStatusChecks,restrictsPushes,restrictsReviewDismissals}}}",
  "variables": {
   "id": "MDIwOkJyYW5jaFByb3RlY3Rpb25SdWxlMTgwMjU3Njc="
  }
}

jcudit avatar Oct 22 '20 19:10 jcudit

+1 – thanks to everyone taking this issue seriously. We also manage 55 repos, and I was hoping to one day manage about 400, but this has slowed us down too much to consider upgrading past 2.9.2.

FWIW a targeted plan of a single repo resources and with five branch protections, WITHOUT refresh and WITHOUT lock, took 15 seconds of real time:

Executed in   15.68 secs   fish           external
   usr time    3.01 secs  111.00 micros    3.01 secs
   sys time    1.36 secs  1250.00 micros    1.36 secs
took 15s

I'd love for graphql to cut the thousands and thousands of REST calls we do per plan down, but not for such an increase in time. I suspect some of this work may be moot/pre-optimization given the refresh improvements in 0.14.0 (as I understand them)–perhaps we could go back to REST entirely for these calls until the issues are sorted?

A full plan without refresh and lock took:

Executed in  207.09 secs   fish           external
   usr time   10.18 secs  120.00 micros   10.18 secs
   sys time    5.01 secs  1060.00 micros    5.01 secs
took 3m27s

And a full plan with refresh and lock:

Executed in  524.93 secs   fish           external
   usr time   17.41 secs  163.00 micros   17.41 secs
   sys time   10.02 secs  1487.00 micros   10.02 secs
took 8m44s

For comparison, a full plan with refresh and lock in 2.9.2:

Executed in  309.87 secs   fish           external
   usr time   15.15 secs  107.00 micros   15.15 secs
   sys time    8.19 secs  893.00 micros    8.19 secs
took 5m9s

So on my machine, 3.0+ is 42% slower in a big plan, and in my testing, 300% slower in a single repo with five branches to protect. For reference, I'm on a 2019 MacBook Pro, maxed out, on a 75MB symmetric connection.

jspiro avatar Nov 12 '20 10:11 jspiro

Can we restore the original codepath that did not use graphql? Many of us are going to be abandoned in v2.9.2 as a result of this performance problem. I'm not sure what benefit graphql has to the end user other, I'm sure it'll be faster someday without the hundreds of REST calls, but it's a noop at best from my perspective as a user, and github's problem not "mine" as long as we stay under the API call limit.

Sorry to be grumpy but this is a pretty serious issue. Terraform sometimes times out waiting.

jspiro avatar Dec 22 '20 03:12 jspiro

Hi all!

To chime in with some data. We are managing some 400 repositories with branch protection on nearly all of them. A refresh takes about 9 minutes which is becoming problematic in our ci/cd pipelines. I notice the branch protection performance issue https://github.com/terraform-providers/terraform-provider-github/issues/565 is still open.

We are currently on version v4.0.1. Has the situation improved in v4.1.0 (or in upcoming versions)? Or should I for now revert to v2.9.x to get better performance? Or should we move to the github_branch_protection_v3 version scheduled for v4.2.0 in https://github.com/terraform-providers/terraform-provider-github/pull/642?

For ref, we run a python script on the side to check we are not missing any projects, which basically fetches the complete config of all repos. That script is able to do so in a couple of seconds. I'm not sure if there is an option in providers to do fetch all the config at once and then cache/re-use. If so, that could greatly improve performance.

Adding some perf test data with terraform v0.13.5 and provider v4.0.1 with 50 repositories:

  • Repositories without team config and branch protection: terraform refresh => 1.96s user 0.91s system 11% cpu 25.865 total
  • Repositories with one team configured: terraform refresh => 2.30s user 1.24s system 9% cpu 37.957 total
  • Repositories with one team configured and one branch protection rule: terraform refresh => 2.87s user 1.65s system 4% cpu 1:37.27 total

In our prod env we have about 400 repositories with typically have 1 branch protection and 3 teams configured.

schans avatar Jan 05 '21 23:01 schans

I'm really hoping #395 will be prioritized, as aside from the security improvements it would allow for a combinatoric decrease in the number of resources needed to manage a large number of repos.

eriksw avatar Jan 29 '21 19:01 eriksw

We're seeing it take ~10 minutes for our 331 repos plus associated Buildkite pipelines:

github v4.10.1

$ terraform state list | cut -d. -f3 | sort | uniq -c
    21
    166 buildkite_pipeline
     27 buildkite_pipeline_schedule
    169 github_branch_default
    331 github_repository
    159 github_repository_webhook
      5 github_team
     18 github_team_membership
    819 github_team_repository

Adding -parallelism=50 to terraform plan didn't seem to make a difference.

Edit: That's ~1.81s per repo, which doesn't seem like much, but it adds up

Edit 2: I've realised we're on an older version of Terraform, v0.14.8. I'll try on the latest

Edit 3: No dice; just as slow on Terraform v0.15.5

ZimbiX avatar Jun 04 '21 08:06 ZimbiX

Hi all,

Recently, I've just moved the pipeline execution to GitHub Actions and have seen a significant improvement in performance.

Previously it was taking 5+ minutes to execute a plan for 120 repositories, but now it's down to just over 1 minute. I also upgraded the provider from v2.9.2 to 4, and have changed from github_branch_protection to github_branch_protection_v3.

josh-barker avatar Sep 07 '21 01:09 josh-barker

@josh-barker what do you mean with

Recently, I've just moved the pipeline execution to GitHub Actions and have seen a significant improvement in performance.

Do you mean that you run terraform steps inside github actions now and that has better performance? What were you using before?

entropitor avatar Sep 07 '21 06:09 entropitor

Hey @entropitor,

Yep, I'm now running the terraform steps inside Github Actions, which has improved the performance. Previously it was running on Azure DevOps agents.

josh-barker avatar Sep 07 '21 07:09 josh-barker

Seeing this as well. Have a repo where we manage 200+ repositories and the refresh alone takes 20 minutes. Using github actions and version = "4.11.0"

dreinhardt89 avatar Dec 07 '21 16:12 dreinhardt89

We're currently close to 500 repos and on the point of moving away from using terraform to manage because runs take over 10 minutes.

The only way I can see this dramatically improve is if all data if fetched at once from the GitHub (GraphQL) api (and cached), because the large amount api calls will always be slow. We have a simple python script that checks for repos not managed by terraform that does this and that runs in 16s. But I don't know if this approach breaks the "terraform model" 🤷

schans avatar Dec 07 '21 16:12 schans

Ya this has been a challenge with the provider from the very beginning. I had to design my states around things like aggressive rate limits, poorly optimized queries, lack of cache, etc on top of larger organization scale. I think the best solution at the moment is to split into smaller states based on some criteria such as team ownership.

majormoses avatar Dec 11 '21 03:12 majormoses

We ended up doing something similar to improve performance, splitting by provider / alphabet for aws: image

mwarkentin avatar Dec 13 '21 16:12 mwarkentin

Hi all,

To share our current status as this might work for others as well. We have implemented a workaround or maybe better labelled a trade-off and for us this solution is now "fast" for daily operations without the need to split up in multiple projects.

What we did:

  • split to the refresh to a separate task and run that through a cron job a couple of times per day (3 or 4). This is the task that takes over 10 minutes for us currently but we're planning to add another couple of hundred repos
  • run the plan and apply with "-refresh=false". These jobs now take around 10 to 30 seconds

The trade-off here is that there is a risk that the state is not up to date and the plan and/or apply might run into errors. For us this is acceptable since:

  • we do not allow direct changes in Github
  • use a s3 state backend
  • have an automated process with ci/cd to run the refresh, plan and apply tasks

HTH

schans avatar Jan 04 '22 21:01 schans

One thing that worked from me and reduced the time from 10min to 1.5min to plan/apply (50repos - 3teams/repo) was to remove the data "github_team".

@mwarkentin use the team slug directly instead of doing a data and using it's output.

I know it's a good approach to use the "data" as a safety measure but in this case it will reduce the plan/apply in 90%

resource "github_team_repository" "****" {
  team_id    = <team_slug>
  repository = github_repository.main.name
  permission = "admin"
}

RuiSMagalhaes avatar Mar 31 '22 10:03 RuiSMagalhaes

@RuiSMagalhaes THANK YOU ❤️ I can confirm this is the case.

I don't get why the data source is so slow but this helps a lot 😍 🚤 🐎

entropitor avatar Mar 31 '22 10:03 entropitor

Anybody tried using github_organization_teams + some logic in locals instead of a bunch of individual data.github_team objects?

jcogilvie avatar Apr 05 '22 17:04 jcogilvie

On our side, we reference teams with the slug directly instead of the data source with our fork of the provider (#1020). Still a bit slow but a lot faster than using data.github_team

dion-gionet avatar Apr 05 '22 19:04 dion-gionet

Update: using github_organization_teams with some transformation logic in my locals provides a similar reduction to my build time (assuming that it's only called once) compared to many individual github_team objects.

Map indexed by team slug:

data "github_organization_teams" "all_teams" {}

locals {
  github_teams = {
    for team in data.github_organization_teams.all_teams.teams : team.slug => team
  }
}

jcogilvie avatar Apr 05 '22 21:04 jcogilvie

Update: using github_organization_teams with some transformation logic in my locals provides a similar reduction to my build time (assuming that it's only called once) compared to many individual github_team objects.

Map indexed by team slug:

data "github_organization_teams" "all_teams" {}

locals {
  github_teams = {
    for team in data.github_organization_teams.all_teams.teams : team.slug => team
  }
}

thanks! this make our plan 3x faster

vaclav-dvorak avatar Aug 12 '22 13:08 vaclav-dvorak

I manage about ~100 repos in my personal account, and I keep hitting rate limit issues, I'm using:

github_repository, github_branch_default, github_branch_protection, github_actions_secret only...

anybody figured work arounds / optimizations for the rate limit issues? this was never an issue before the GraphQL refactor of this provider...

ahmadnassri avatar Aug 24 '22 15:08 ahmadnassri

👋 Hey Friends, this issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Please add the Status: Pinned label if you feel that this issue needs to remain open/active. Thank you for your contributions and help in keeping things tidy!

github-actions[bot] avatar May 22 '23 01:05 github-actions[bot]

This is very much still a thing, we recently rev'd to latest in our infra and it seems to have only gotten worse :(

shoddyguard avatar May 22 '23 09:05 shoddyguard