flux-iac
Error: reading token file: reading OIDC Token
Error: reading token file: reading OIDC Token
Config Definition
I am having some issues performing auth to Azure when using the runner. My setup is detailed below which detailed how my setup and config looks like
apiVersion: infra.contrib.fluxcd.io/v1alpha2
kind: Terraform
metadata:
name: azure-resources-tf-sa1
namespace: flux-system
spec:
interval: 2m
approvePlan: auto
path: ./Terraform
tfstate:
forceUnlock: auto
backendConfig:
customConfiguration: |
backend "azurerm" {
resource_group_name = "d-aks-rg"
storage_account_name = "storageaccount"
container_name = "tfstate"
key = "azure-resources.tfstate"
use_oidc = true
}
sourceRef:
kind: GitRepository
name: dnm
namespace: flux-system
serviceAccountName: azure-tf-controller-sa
runnerPodTemplate:
metadata:
labels:
aadpodidbinding: azure-tf-controller-identity
annotations:
azure.workload.identity/client-id: "xxxx-xxxx-xxxxx-xxxx"
spec:
image: ghcr.io/weaveworks/tf-runner-azure:v0.15.1 #ghcr.io/weaveworks/tf-runner:latest
env:
- name: ARM_USE_OIDC
value: "true"
- name: ARM_SUBSCRIPTION_ID
value: "xxxx-xxxx-xxxxx-xxxx"
- name: ARM_TENANT_ID
value: "xxxx-xxxx-xxxxx-xxxx"
- name: ARM_CLIENT_ID
value: "xxxx-xxxx-xxxxx-xxxx"
- name: ARM_OIDC_TOKEN_FILE_PATH
value: "/var/run/secrets/azure/tokens/azure-identity-token"
# tolerations:
# - key: "terraform"
# operator: "Equal"
# value: "true"
# effect: "NoSchedule"
# nodeSelector:
# purpose: terraform
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
name: dnm
namespace: flux-system
spec:
interval: 30s
url: https://github.com/xxxx-xxxx-xxxxx-xxxx/repo
ref:
branch: main
apiVersion: v1
kind: ServiceAccount
metadata:
name: azure-tf-controller-sa
namespace: flux-system
annotations:
azure.workload.identity/client-id: "xxxx-xxxx-xxxxx-xxxx"
azure.workload.identity/tenant-id: "xxxx-xxxx-xxxxx-xxxx"
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: terraform-runner-role
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["infra.contrib.fluxcd.io"]
resources: ["*"]
verbs: ["*"]
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: terraform-runner-rolebinding
subjects:
- kind: ServiceAccount
name: azure-tf-controller-sa
namespace: flux-system
roleRef:
kind: ClusterRole
name: terraform-runner-role
apiGroup: rbac.authorization.k8s.io
Error Log
The error noticed in the runner:
{"level":"info","ts":"2024-11-01T15:29:15.626Z","logger":"runner.terraform","msg":"generating the template founds"}
{"level":"info","ts":"2024-11-01T15:29:15.626Z","logger":"runner.terraform","msg":"main.tf.tpl not found, skipping"}
{"level":"info","ts":"2024-11-01T15:29:15.626Z","logger":"runner.terraform","msg":"initializing","instance-id":"2fc52e85-d74a-4bd6-84e7-8ff9a0487cd6"}
{"level":"info","ts":"2024-11-01T15:29:15.626Z","logger":"runner.terraform","msg":"mapping the Spec.BackendConfigsFrom","instance-id":"2fc52e85-d74a-4bd6-84e7-8ff9a0487cd6"}
{
"terraform_version": "1.3.9",
"platform": "linux_amd64",
"provider_selections": {},
"terraform_outdated": true
}
Upgrading modules...
- aks in modules/aks
Initializing the backend...
Error: reading token file: reading OIDC Token "/var/run/secrets/azure/tokens/azure-identity-token": open /var/run/secrets/azure/tokens/azure-identity-token: no such file or directory
{"level":"error","ts":"2024-11-01T15:29:15.787Z","logger":"runner.terraform","msg":"unable to initialize","instance-id":"2fc52e85-d74a-4bd6-84e7-8ff9a0487cd6","error":"exit status 1
Error: reading token file: reading OIDC Token "/var/run/secrets/azure/tokens/azure-identity-token": open /var/run/secrets/azure/tokens/azure-identity-token: no such file or directory
"}
{"level":"info","ts":"2024-11-01T15:29:15.787Z","logger":"runner.terraform","msg":"cleanup TmpDir","instance-id":"2fc52e85-d74a-4bd6-84e7-8ff9a0487cd6","tmpDir":"/tmp/flux-system-azure-resources-tf-sa1"}
Notes
Additional Context I am using Azure Pod Identity Binding and have created a federated identity on the managed identity with access to the Kubernetes cluster.
Possible Issue The error suggests that the OIDC token file at /var/run/secrets/azure/tokens/azure-identity-token is missing, which is likely related to the Azure Pod Identity setup. I have configured the Service Account with appropriate annotations for Azure Workload Identity, but it appears the runner cannot access the required token file.
Steps Taken Verified the Service Account and managed identity setup. Ensured aadpodidbinding and other annotations are correctly configured. Any insights or guidance on resolving the missing OIDC token file issue would be greatly appreciated.
