ibm_is_vpn_gateway: provider crash during terraform apply

[toddgiguere]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

• IBM provider v1.58.1 and v1.59.0

Affected Resource(s)

• ibm_is_vpn_gateway

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "ibm_is_vpn_gateway" "gateway" {
  for_each       = local.vpn_gateway_map
  name           = "${var.prefix}-${each.key}"
  subnet         = each.value.subnet_id
  mode           = each.value.mode
  resource_group = each.value.resource_group == null ? null : local.resource_groups[each.value.resource_group]
  tags           = var.tags
  access_tags    = each.value.access_tags

  timeouts {
    delete = "1h"
  }
}

Part of larger DA project: https://github.com/terraform-ibm-modules/terraform-ibm-landing-zone/tree/main

Debug Output

module.vsi_landing_zone.module.landing_zone.ibm_is_vpn_gateway.gateway["management-gateway"]: Still creating... [3m10s elapsed]
 │ Error: Plugin did not respond
 │ 
 │   with module.vsi_landing_zone.module.landing_zone.ibm_is_vpn_gateway.gateway["management-gateway"],
 │   on ../../vpn.tf line 17, in resource "ibm_is_vpn_gateway" "gateway":
 │   17: resource "ibm_is_vpn_gateway" "gateway" {
 │ 
 │ The plugin encountered an error, and failed to respond to the
 │ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
 │ contain more details.
 
  Stack trace from the terraform-provider-ibm_v1.58.1 plugin:
 panic: runtime error: invalid memory address or nil pointer dereference
 [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x36ae0bd]

Panic Output

goroutine 2419 [running]:
github.com/IBM-Cloud/terraform-provider-ibm/ibm/service/vpc.vpngwGet(0xc002421c80, {0x439a720?, 0xc0024ca000}, {0xc001af7cb0, 0x29})
	github.com/IBM-Cloud/terraform-provider-ibm/ibm/service/vpc/resource_ibm_is_vpn_gateway.go:493 +0x99d
github.com/IBM-Cloud/terraform-provider-ibm/ibm/service/vpc.resourceIBMISVPNGatewayRead(0xc002421c80, {0x439a720, 0xc0024ca000})
	github.com/IBM-Cloud/terraform-provider-ibm/ibm/service/vpc/resource_ibm_is_vpn_gateway.go:424 +0xc8
github.com/IBM-Cloud/terraform-provider-ibm/ibm/service/vpc.resourceIBMISVPNGatewayCreate(0x0?, {0x439a720, 0xc0024ca000})
	github.com/IBM-Cloud/terraform-provider-ibm/ibm/service/vpc/resource_ibm_is_vpn_gateway.go:325 +0x13b
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0x4c74d70?, {0x4c74d70?, 0xc001b9ade0?}, 0xd?, {0x439a720?, 0xc0024ca000?})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:695 +0x178
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0xc001002ee0, {0x4c74d70, 0xc001b9ade0}, 0xc000d80270, 0xc002421800, {0x439a720, 0xc0024ca000})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:837 +0xa7a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0xc0009644b0, {0x4c74d70?, 0xc001b9acc0?}, 0xc000894af0)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:1021 +0xe3c
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0xc0008ce640, {0x4c74d70?, 0xc001b9a660?}, 0xc00157d730)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:818 +0x574
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0x4205c20?, 0xc0008ce640}, {0x4c74d70, 0xc001b9a660}, 0xc00157d6c0, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:385 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000000960, {0x4c7cb40, 0xc00124e820}, 0xc0015f7560, 0xc001215740, 0x6a87720, 0x0)
	google.golang.org/[email protected]/server.go:1360 +0xe13
google.golang.org/grpc.(*Server).handleStream(0xc000000960, {0x4c7cb40, 0xc00124e820}, 0xc0015f7560, 0x0)
	google.golang.org/[email protected]/server.go:1737 +0xa1b
google.golang.org/grpc.(*Server).serveStreams.func1.1()
	google.golang.org/[email protected]/server.go:982 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/[email protected]/server.go:980 +0x18c

Error: The terraform-provider-ibm_v1.58.1 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

TestRunVSIPatternWithHPCS 2023-10-31T06:19:27Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; ╷
│ Error: Plugin did not respond

Expected Behavior

Apply without error

Actual Behavior

Cannot reproduce at will.

This panic error (provider crash) happens intermittently, and is not consistent. We have observed this error in our CI process, so the variables submitted to this terraform block are not changing between a successful run and one that causes this panic.

Steps to Reproduce

1. terraform apply

Important Factoids

Can provide full terraform trace log if required (very large)

References

• #0000

[ocofaigh]

Also facing this issue VERY frequently in the past few days when running the terraform code in here -> https://github.com/terraform-ibm-modules/terraform-ibm-landing-zone/tree/main/patterns

[ujjwal-ibm]

looking at it

[ocofaigh]

Looks like PR for this was merged https://github.com/IBM-Cloud/terraform-provider-ibm/pull/4903 ?

[ocofaigh]

Looks like fix is in https://github.com/IBM-Cloud/terraform-provider-ibm/releases/tag/v1.60.0 - can we close this issue?

[deepaksibm]

As per above comment requesting to close the issue @sumitkumartiwari