terraform-provider-oci
terraform-provider-oci copied to clipboard
oci_network_load_balancer_backend 404-NotAuthorizedOrNotFound but the backeds are correctly created
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Terraform Version and Provider Version
Terraform v1.1.6 on linux_amd64
- provider registry.terraform.io/hashicorp/oci v4.64.0
- provider registry.terraform.io/hashicorp/oci v4.76.0
- provider oracle/oci 4.76.0
- provider registry.terraform.io/hashicorp/template v2.2.0
i've tried also oracle/oci module v.4.64.0 and the latest oracle oci module
Affected Resource(s)
oci_network_load_balancer_backend
Terraform Configuration Files
lb.tf
resource "oci_network_load_balancer_network_load_balancer" "k3s_load_balancer" {
compartment_id = var.compartment_ocid
display_name = var.k3s_load_balancer_name
subnet_id = oci_core_subnet.oci_core_subnet11.id
is_private = true
is_preserve_source_destination = false
freeform_tags = {
"provisioner" = "terraform"
"environment" = "${var.environment}"
"${var.unique_tag_key}" = "${var.unique_tag_value}"
}
}
resource "oci_network_load_balancer_listener" "k3s_kube_api_listener" {
default_backend_set_name = oci_network_load_balancer_backend_set.k3s_kube_api_backend_set.name
name = "k3s kube api listener"
network_load_balancer_id = oci_network_load_balancer_network_load_balancer.k3s_load_balancer.id
port = var.kube_api_port
protocol = "TCP"
}
resource "oci_network_load_balancer_backend_set" "k3s_kube_api_backend_set" {
health_checker {
protocol = "TCP"
port = var.kube_api_port
}
name = "k3s kube api backend"
network_load_balancer_id = oci_network_load_balancer_network_load_balancer.k3s_load_balancer.id
policy = "FIVE_TUPLE"
is_preserve_source = true
}
resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
depends_on = [
oci_core_instance_pool.k3s_servers,
]
count = 2
backend_set_name = oci_network_load_balancer_backend_set.k3s_kube_api_backend_set.name
network_load_balancer_id = oci_network_load_balancer_network_load_balancer.k3s_load_balancer.id
port = var.kube_api_port
target_id = data.oci_core_instance_pool_instances.k3s_servers_instances.instances[count.index].id
}
data.tf
data "oci_core_instance_pool_instances" "k3s_servers_instances" {
depends_on = [
oci_core_instance_pool.k3s_servers,
]
compartment_id = var.compartment_ocid
instance_pool_id = oci_core_instance_pool.k3s_servers.id
}
instance_pool.tf
resource "oci_core_instance_pool" "k3s_servers" {
depends_on = [
oci_identity_dynamic_group.compute_dynamic_group,
oci_identity_policy.compute_dynamic_group_policy
]
lifecycle {
create_before_destroy = true
ignore_changes = [load_balancers, freeform_tags]
}
display_name = "k3s-servers"
compartment_id = var.compartment_ocid
instance_configuration_id = oci_core_instance_configuration.k3s_server_template.id
placement_configurations {
availability_domain = var.availability_domain
primary_subnet_id = oci_core_subnet.default_oci_core_subnet10.id
fault_domains = var.fault_domains
}
size = 2
freeform_tags = {
"provisioner" = "terraform"
"environment" = "${var.environment}"
"${var.unique_tag_key}" = "${var.unique_tag_value}"
"k3s-cluster-name" = "${var.cluster_name}"
"k3s-instance-type" = "k3s-server"
}
}
Panic Output
│ Error: 404-NotAuthorizedOrNotFound, Unknown resource Entity of type Backend with key ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycg2zjcqxbsbe26rwvc6szowce7u7qpi2m5gaqxm53bwza.6443 not found
│ Suggestion: Either the resource has been deleted or service Network Load Balancer Backend need policy to access this resource. Policy reference: https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/network_load_balancer_backend
│ Request Target: GET https://network-load-balancer-api.eu-zurich-1.oci.oraclecloud.com/20200501/networkLoadBalancers/ocid1.networkloadbalancer.oc1.eu-zurich-1.amaaaaaa5kjm7pyao4axnaunpvssqzrljdopyyuouepvl5dlvsd5d4y7q3ya/backendSets/k3s%20kube%20api%20backend/backends/ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycg2zjcqxbsbe26rwvc6szowce7u7qpi2m5gaqxm53bwza.6443
│ Provider version: 4.76.0, released on 2022-05-21.
│ Service: Network Load Balancer Backend
│ Operation Name: GetBackend
│ OPC request ID: b55d15500a84243335655841e09c3ab3/2B8A5720ED91321A19ADF40B894FAD48/AC30D14BAA69E1C642354767A3470EA3
│
│
│ with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0],
│ on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│ 36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
│
╵
╷
│ Error: 404-NotAuthorizedOrNotFound, Unknown resource Entity of type Backend with key ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycfdk2fs3stowtrhca2dmtz2auu6rkx5qlwfrqxurxlhaq.6443 not found
│ Suggestion: Either the resource has been deleted or service Network Load Balancer Backend need policy to access this resource. Policy reference: https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/network_load_balancer_backend
│ Request Target: GET https://network-load-balancer-api.eu-zurich-1.oci.oraclecloud.com/20200501/networkLoadBalancers/ocid1.networkloadbalancer.oc1.eu-zurich-1.amaaaaaa5kjm7pyao4axnaunpvssqzrljdopyyuouepvl5dlvsd5d4y7q3ya/backendSets/k3s%20kube%20api%20backend/backends/ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycfdk2fs3stowtrhca2dmtz2auu6rkx5qlwfrqxurxlhaq.6443
│ Provider version: 4.76.0, released on 2022-05-21.
│ Service: Network Load Balancer Backend
│ Operation Name: GetBackend
│ OPC request ID: 521d57cf4abd2ac90d40ac9d7c1a6ded/8B5292856DD61813672E31F59476D54D/74ABF142B8425A581854E9E355706EC8
│
│
│ with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[1],
│ on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│ 36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
│
If i try to apply again:
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [2m41s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [2m51s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m1s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m11s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m21s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m31s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m41s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m51s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [4m1s elapsed]
╷
│ Error: 409-NotAuthorizedOrResourceAlreadyExists, Conflict. For nlb-id Duplicate backend IP/id + port combinations not allowed: Backend(name=null, ipVersion=Ipv4, ipAddress=null, targetId=ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycfdk2fs3stowtrhca2dmtz2auu6rkx5qlwfrqxurxlhaq, port=6443, weight=1, isDrain=false, isBackup=false, isOffline=false)
│ Suggestion: The resource is in a conflicted state. Please retry again or contact support for help with service: Network Load Balancer Backend
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/network_load_balancer_backend
│ Request Target: POST https://network-load-balancer-api.eu-zurich-1.oci.oraclecloud.com/20200501/networkLoadBalancers/ocid1.networkloadbalancer.oc1.eu-zurich-1.amaaaaaa5kjm7pyao4axnaunpvssqzrljdopyyuouepvl5dlvsd5d4y7q3ya/backendSets/k3s%20kube%20api%20backend/backends
│ Provider version: 4.76.0, released on 2022-05-21.
│ Service: Network Load Balancer Backend
│ Operation Name: CreateBackend
│ OPC request ID: de113a2f981f3412fc646ea3667b1edf/42712CB6BCC726E45949054091989B81/70843B5EC104486CD24F74190B741DC2
│
│
│ with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[1],
│ on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│ 36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
│
╵
╷
│ Error: 409-NotAuthorizedOrResourceAlreadyExists, Conflict. For nlb-id Duplicate backend IP/id + port combinations not allowed: Backend(name=null, ipVersion=Ipv4, ipAddress=null, targetId=ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycg2zjcqxbsbe26rwvc6szowce7u7qpi2m5gaqxm53bwza, port=6443, weight=1, isDrain=false, isBackup=false, isOffline=false)
│ Suggestion: The resource is in a conflicted state. Please retry again or contact support for help with service: Network Load Balancer Backend
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/network_load_balancer_backend
│ Request Target: POST https://network-load-balancer-api.eu-zurich-1.oci.oraclecloud.com/20200501/networkLoadBalancers/ocid1.networkloadbalancer.oc1.eu-zurich-1.amaaaaaa5kjm7pyao4axnaunpvssqzrljdopyyuouepvl5dlvsd5d4y7q3ya/backendSets/k3s%20kube%20api%20backend/backends
│ Provider version: 4.76.0, released on 2022-05-21.
│ Service: Network Load Balancer Backend
│ Operation Name: CreateBackend
│ OPC request ID: 50e0fe689671c85f94d5001fa64469b8/62921F9F81DA9FCC625A918BF0F6872C/8F2367F2069564F8E1629E7A7A0DB7C4
│
│
│ with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0],
│ on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│ 36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
│
Actual Behavior
Backend are correctly created but 404-NotAuthorizedOrNotFound error occured
Steps to Reproduce
Apply this module
The same probelm occures if i try to use the private ip ocid as target_id:
│ Error: 404-NotAuthorizedOrNotFound
│ Provider version: 4.64.0, released on 2022-02-16. This provider is 13 Update(s) behind to current.
│ Service: Network Load Balancer Backend
│ Error Message: Unknown resource Entity of type Backend with key ocid1.privateip.oc1.eu-zurich-1.ab5heljrku3dvhwandbamb34s7cthenz4vki52iycqgoj5obfkoexbserhzq.6443 not found
│ OPC request ID: 73aca760d408945945bcb3ed681d8b8b/EF6781C100C57DB2EF6AC79D40FF19F1/74E7BA9E972DCF1A125522BF258F90E0
│ Suggestion: Either the resource has been deleted or service Network Load Balancer Backend need policy to access this resource. Policy reference: https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm
│
│
│ with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0],
│ on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│ 36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
│
╵
╷
│ Error: 404-NotAuthorizedOrNotFound
│ Provider version: 4.64.0, released on 2022-02-16. This provider is 13 Update(s) behind to current.
│ Service: Network Load Balancer Backend
│ Error Message: Unknown resource Entity of type Backend with key ocid1.privateip.oc1.eu-zurich-1.ab5heljryssggfe4rldf26asmvcpjrj2aoktk7fp6yzwdxpngqszpqgugfja.6443 not found
│ OPC request ID: ba08ecc04d664aa6a44161e9fa453ec3/06469C6E7FA4A2840278859A5C74EB5E/79880A506DB3F22E3DA470E79C826946
│ Suggestion: Either the resource has been deleted or service Network Load Balancer Backend need policy to access this resource. Policy reference: https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm
│
│
│ with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[1],
│ on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│ 36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
I've found a workaround. With some reverse enginereeng i've inspected the html of the web console and i've found that the name was setted to the instance name (the name if is not provided is automatically generated). Inspecting the page i've found that in the id of the html the name instead was setted to $instance_id:$backend_port
Then i've tried to import the resurce with:
terraform import module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[1] "networkLoadBalancers/ocid1.networkloadbalancer.oc1.eu-zurich-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/backendSets/k3s_kube_api_backend/backends/ocid1.instance.oc1.eu-zurich-1.xxxxxxxxxxxxxxxxxxxxxx:6443"
and the import has correctly imported the backend
Changing the backend name to $instance_id:$backend_port fix the problem, but this is only a workaround.
Also, with this workaround the backand name in the web console is always setted to the instance name. I think something has changed in the OCI api and at the moment is not documented.
UPDATE
The value in the name argument can be any kind of string
name = format("%s_%s", "k3s_server", count.index)
or
name = data.oci_core_instance_pool_instances.k3s_servers_instances.instances[count.index].display_name
so the name argument is a required argument, the value must be "any kind of string". In the web interface the result is always the same, the instance name is displayed:
This is the working code:
resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
depends_on = [
oci_core_instance_pool.k3s_servers,
]
count = var.k3s_server_pool_size
backend_set_name = oci_network_load_balancer_backend_set.k3s_kube_api_backend_set.name
network_load_balancer_id = oci_network_load_balancer_network_load_balancer.k3s_load_balancer.id
name = data.oci_core_instance_pool_instances.k3s_servers_instances.instances[count.index].display_name
port = var.kube_api_port
target_id = data.oci_core_instance_pool_instances.k3s_servers_instances.instances[count.index].id
}
Thank you for reporting the issue. We have raised an internal ticket to track this. Our service engineers will get back to you.