pipeline icon indicating copy to clipboard operation
pipeline copied to clipboard

Tekton shared workspace task between TF action init plan and apply

Open tppalani opened this issue 1 year ago • 0 comments

Expected Behavior

Tekton workspace should be shared between other task from k8s PVC.

Actual Behavior

I have tekton pipeline task to clone github source code as part of intial task then followed with clone i have created 3 different task for terraform init, plan, auto apply, i have create 3 different for each action.

Now git cloned data's storing the repo data as part of shared workspace. Now the tf init task which contains all the tf configuration to autheicate to private registry to downlond the requried tf modules all the things working as we expected (init, plan, apply) but tekton task i have written not reusable which means i just copy pasted all the 3 task configuration data into new files but code remains will be the same unless tf init, plan, apply.

Now to avoid the same code enter into 3 tf task it tf workspace needs to share the tf configuration data across 3 tf task. this would be keep the tekton task clean without adding duplciate code.

Steps to Reproduce the Problem

pipeline.yaml

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: build-and-deploy-pipeline
spec:
  workspaces:
    - name: pipeline-ws
    - name: task-ws
  params:
    - name: url
    - name: revision
      default:  java-pipeline
    - name: subdirectory
  tasks:
    - name: scm-checkout
      taskRef:
        name: git-clone
      params:
        - name: url
          value: $(params.url)
        - name: revision
          value: $(params.revision)
        - name: subdirectory
          value: $(params.subdirectory)
        - name: deleteExisting
          value: "true"
      workspaces:
        - name: output
          workspace: pipeline-ws
 
    - name: stage-terraform-init
      taskRef:
        name: stage-terraform-init
      runAfter:
        - scm-checkout
      workspaces:
        - name: task-ws
          workspace: task-ws

run.yaml

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  generateName: dev-pipeline
spec:
  params:
    - name: url
      value: https://my-source-code.git
    - name: subdirectory
      value: source
    - name: revision
      value: "development"
  pipelineRef:
    name: build-and-deploy-pipeline
  podTemplate:
    securityContext:
      runAsUser: 0
      fsGroup: 1001
  workspaces:
    - name: task-ws
      persistentVolumeClaim:
        claimName: tf-configuration-pvc
    - name: pipeline-ws
      persistentVolumeClaim:
        claimName: dev-pipeline-pvc

tf init task

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: stage-terraform-init
  namespace: "dev-pipeline"
spec:
  workspaces:
    - name: task-ws
      description: Workspace for sharing files between tasks
  params:
    - name: tf-secret-name
      default: tf-secret
      description: this is the secrets name which contains the .terraformrc file
      type: string
    
    - name: ARGS
      description: The terraform cli commands to tun
      type: array
      default:
      - "--help"

    - name: tf-workspaces
      description: The terraform workspace which will be used for deployment
      type: string
      default: dev

    - name: image
      default: kubectl-aws
      type: string
  steps:
    - name:  stage-terraform-init
      image: $(params.image)
      workingDir: $(workspaces.task-ws.path)
      volumeMounts:
        - name: my-tf-secret-volume
          mountPath: /tmp/.terraformrc
          subPath: .terraformrc
      
      script: |
        #!/usr/bin/env sh
         pwd
         ls -lrt /workspace/task-ws
         echo "Starting Terraform init execution"
         cp -r /workspace/task-ws/source/src /workspace/task-ws/source/infra
         cd /workspace/task-ws/source/INFRA
         wget https://releases.hashicorp.com/terraform/1.0.11/terraform_1.0.11_linux_amd64.zip --no-check-certificate
         unzip terraform_1.0.11_linux_amd64.zip
         chmod +x ./terraform
         mkdir -p $HOME/bin && cp ./terraform $HOME/bin/terraform && export PATH=$PATH:$HOME/bin
         echo $PATH
         cp  /tmp/.terraformrc ~/.terraformrc
         cp ./$(params.tf-workspaces)/$(params.tf-workspaces)-provider.tf $(params.tf-workspaces)-provider.tf
         cp ./$(params.tf-workspaces)/$(params.tf-workspaces).auto.tfvars $(params.tf-workspaces).auto.tfvars
         cp ./$(params.tf-workspaces)/security-group.tf $(params.tf-workspaces)-security-group.tf
         terraform init
         echo "completed Terraform init execution"
        
  volumes:
    - name: my-tf-secret-volume
      secret:
        secretName: $(params.tf-secret-name)

tf task plan

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: stage-terraform-init
  namespace: "dev-pipeline"
spec:
  workspaces:
    - name: task-ws
      description: Workspace for sharing files between tasks
  params:
    - name: tf-secret-name
      default: tf-secret
      description: this is the secrets name which contains the .terraformrc file
      type: string
    
    - name: ARGS
      description: The terraform cli commands to tun
      type: array
      default:
      - "--help"

    - name: tf-workspaces
      description: The terraform workspace which will be used for deployment
      type: string
      default: dev

    - name: image
      default: kubectl-aws
      type: string
  steps:
    - name:  stage-terraform-init
      image: $(params.image)
      workingDir: $(workspaces.task-ws.path)
      volumeMounts:
        - name: my-tf-secret-volume
          mountPath: /tmp/.terraformrc
          subPath: .terraformrc
      
      script: |
        #!/usr/bin/env sh
         pwd
         ls -lrt /workspace/task-ws
         echo "Starting Terraform init execution"
         cp -r /workspace/task-ws/source/src /workspace/task-ws/source/infra
         cd /workspace/task-ws/source/INFRA
         wget https://releases.hashicorp.com/terraform/1.0.11/terraform_1.0.11_linux_amd64.zip --no-check-certificate
         unzip terraform_1.0.11_linux_amd64.zip
         chmod +x ./terraform
         mkdir -p $HOME/bin && cp ./terraform $HOME/bin/terraform && export PATH=$PATH:$HOME/bin
         echo $PATH
         cp  /tmp/.terraformrc ~/.terraformrc
         cp ./$(params.tf-workspaces)/$(params.tf-workspaces)-provider.tf $(params.tf-workspaces)-provider.tf
         cp ./$(params.tf-workspaces)/$(params.tf-workspaces).auto.tfvars $(params.tf-workspaces).auto.tfvars
         cp ./$(params.tf-workspaces)/security-group.tf $(params.tf-workspaces)-security-group.tf
         terraform plan
         echo "completed Terraform init execution"
        
  volumes:
    - name: my-tf-secret-volume
      secret:
        secretName: $(params.tf-secret-name)

if see my code both tf init and plan has same kind of code how to eliminate this and i want keep only all the configuration in tf init file task and another file tf apply task file should having only tf apply to share the worksapce.

tppalani avatar Feb 04 '24 16:02 tppalani