terraform-k8s
terraform-k8s copied to clipboard
Introduce better logging
Following the guide here
Terraform workspaces created but no run configuration.
Exception:
2021-05-29T00:38:51.072Z INFO terraform-k8s Checking outputs {"Organization": "nslhub", "WorkspaceID": "ws-fkYap2hAd8ogTkxp", "RunID": ""}
2021-05-29T00:38:51.094Z INFO terraform-k8s Updated outputs {"Organization": "nslhub", "WorkspaceID": "ws-fkYap2hAd8ogTkxp"}
2021-05-29T00:38:51.094Z INFO terraform-k8s Updating secrets {"name": "greetings-outputs"}
2021-05-29T00:38:51.094Z DEBUG controller-runtime.manager.events Normal {"object": {"kind":"Workspace","namespace":"ci","name":"greetings","uid":"220cacfe-7f6c-4717-9258-ad02aec80e87","apiVersion":"app.terraform.io/v1alpha1","resourceVersion":"16909471"}, "reason": "WorkspaceEvent", "message": "Updated outputs for run "}
2021-05-29T00:38:51.564Z INFO terraform-k8s Starting module backed run {"Organization": "nslhub", "Name": "greetings", "Namespace": "ci"}
2021-05-29T00:38:51.775Z ERROR controller Reconciler error {"controller": "workspace-controller", "name": "greetings", "namespace": "ci", "error": "resource not found"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:246
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:218
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:197
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.Until
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90
This makes it very tough to understand the issue. Also observed
Status:
Config Version ID:
Run ID:
Run Status:
Workspace ID: ws-fkYap2hAd8ogTkxp
The issue starts resurfacing again,
2021-06-12T07:59:02.740Z ERROR terraform-k8s Failed to update output status {"error": "workspaces.app.terraform.io \"s3-logs-ap-south-1\" not found"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
github.com/hashicorp/terraform-k8s/workspacehelper.(*WorkspaceHelper).reconcileWorkspace
/workspace/workspacehelper/workspace_helper.go:142
github.com/hashicorp/terraform-k8s/workspacehelper.(*WorkspaceHelper).Reconcile
/workspace/workspacehelper/workspace_helper.go:442
github.com/hashicorp/terraform-k8s/controllers.(*WorkspaceReconciler).Reconcile
/workspace/controllers/workspace_controller.go:63
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:244
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:218
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:197
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.Until
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90
On further debugging its failing here https://github.com/hashicorp/terraform-k8s/blob/973ac76005280a5aff9656fcb39a436dcda481c4/workspacehelper/workspace_helper.go#L141
There is no way to identify the cause of this. It seems it is failing when updating the workspace id in workspace resource.
@rverma-nsl Did you ever figure out what was the root cause of your first issue? Facing the same but can't figure out what the issue is...