Recursive Path Selection

[dfreilich]

Hi!

Is there any option to watch all files in a directory, recursively including subdirectories?

I had tried setting up a tf object to deploy redis, and when I had the files as top-level files in my source directory, it successfully deployed. When I moved them into a redis/ subdirectory, it no longer worked, which was not the expected behavior.

My controller spec is as follows (with TF controller version v0.10.0-rc.6):

---
apiVersion: infra.contrib.fluxcd.io/v1alpha1
kind: Terraform
metadata:
  name: staging
  namespace: flux-system
spec:
  approvePlan: "auto"
  destroyResourcesOnDeletion: true
  serviceAccountName: tf-controller
  refreshBeforeApply: true
  interval: 30s
  path: ./
  sourceRef:
    kind: GitRepository
    name: staging
    namespace: flux-system

[dfreilich]

After talking to @bivas, I understood that this was the desired functionality - that there is a need to create a different Terraform object for any specific directory, given that Terraform by it's nature looks at different directories as different entities.

Was that your thought-process?

[chanwit]

Hi @dfreilich

It should work regardless of the path. Maybe there's an absolute path reference inside your Terraform module. There's a similar case when people are trying to deploy a lambda function.

[dfreilich]

To clarify - it should watch all files recursively? The TF Object worked when I changed the path to reflect the new path (the ./redis directory). However, the object which had been watching path: ./ didn't detect files in the ./redis directory.

[chanwit]

it does that. we already have a complex directory structure in this repo for example,

https://github.com/tf-controller/eks-scaling/blob/main/_artifacts/30-tfc-helloworld/tfc-helloworld.yaml

[chanwit]

Maybe there's something in your setup that I don't understand. What are the controller logs and the runner logs saying?

[dfreilich]

The exact scenario I had was where I had deployed a Terraform object in my fleet directory, which was watching path: ./source_dir/.

Then, in the directory, I created a ./source_dir/redis.tf file, pushed it, and the Terraform object successfully saw that there was a tf file, and applied it.

Then, in another commit, I moved the ./source_dir/redis.tf folder into a subdirectory, called redis. My setup was therefore:

source_dir/
|                 | redis/
|                 |           | redis.tf

When I pushed that, the Terraform object detected that something had changed, and only saw that the files were removed from the source_dir/ directory, but it didn't detect the file that was currently in source_dir/redis/redis.tf.

However, when I deployed a Terraform object which was watching path: ./source_dir/redis, it did successfully deploy.

[chanwit]

Ah ok, I understand your problem now.

In the current implementation, you're right, the runner is looking for the module only at 1 level. It's by design as we have to generate many helper files like the backend configuration, the variable, and the cli configuration files.

We have a plan to support workspaces in the future, so this behavior is unlikely to change.