Allow terraform module to specify complex variable structures

[kosalaat]

trafficstars

• Terrform variable types are mapped to ansible veriable types

• Currently handles Dict, List, Str, Int, Bool types

• Updated the documentation accordingly

• Updated with an example.

SUMMARY

Added the capability to specify complex variables structures in the terraform module. With this change we can specify Terraform Object / Nested type variables via ansible:

For example, in TF:

variable "vm_additional_disks" {
  description = "Additional Disks"
  type = list(object({
    unit_number      = number
    size             = number
    label            = string
    thin_provisioned = bool
  }))
  default = []
}

Would translate to:

        vm_additional_disks:
          - label: "Third Disk"
            size: 40
            thin_provisioned: true
            unit_number: 2
          - label: "Fourth Disk"
            size: 22
            thin_provisioned: true
            unit_number: 3

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

terraform

ADDITIONAL INFORMATION

Changed the variable_args to be constructed via a recursive function process_args. It's capable of handling nested lists, dictionaries, bools, string and numbers.

[ansibullbot]

cc @m-yosefpor @rainerleber click here for bot help

[kosalaat]

Thanks for your contribution! Please add a changelog fragment.

Please also note that we had a similar proposal in the past (#4281) which had to be reverted for breaking backwards compatibility. It looks to me like your approach should work better, but I'm not using terraform so I cannot really judge.

Added the change log fragment.

[ansibullbot]

@kosalaat this PR contains the following merge commits:

• https://github.com/ansible-collections/community.general/commit/0ce1ecf587f0357620a9905e25174c2d23ce7826

Please rebase your branch to remove these commits.

click here for bot help

[ansibullbot]

The test ansible-test sanity --test pep8 [explain] failed with 3 errors:

plugins/modules/cloud/misc/terraform.py:388:161: E501: line too long (164 > 160 characters)
plugins/modules/cloud/misc/terraform.py:479:22: E211: whitespace before '('
plugins/modules/cloud/misc/terraform.py:480:18: E231: missing whitespace after ','

The test ansible-test sanity --test pep8 [explain] failed with 3 errors:

plugins/modules/cloud/misc/terraform.py:388:161: E501: line too long (164 > 160 characters)
plugins/modules/cloud/misc/terraform.py:479:22: E211: whitespace before '('
plugins/modules/cloud/misc/terraform.py:480:18: E231: missing whitespace after ','

The test ansible-test sanity --test pep8 [explain] failed with 3 errors:

plugins/modules/cloud/misc/terraform.py:388:161: E501: line too long (164 > 160 characters)
plugins/modules/cloud/misc/terraform.py:479:22: E211: whitespace before '('
plugins/modules/cloud/misc/terraform.py:480:18: E231: missing whitespace after ','

The test ansible-test sanity --test pep8 [explain] failed with 3 errors:

plugins/modules/cloud/misc/terraform.py:388:161: E501: line too long (164 > 160 characters)
plugins/modules/cloud/misc/terraform.py:479:22: E211: whitespace before '('
plugins/modules/cloud/misc/terraform.py:480:18: E231: missing whitespace after ','

click here for bot help

[github-actions[bot]]

Docs Build 📝

Thank you for contribution!✨

This PR has been merged and your docs changes will be incorporated when they are next published.

[ansibullbot]

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:526:10: E231: missing whitespace after ','

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:526:10: E231: missing whitespace after ','

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:526:10: E231: missing whitespace after ','

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:526:10: E231: missing whitespace after ','

click here for bot help

[ansibullbot]

@kosalaat This PR was evaluated as a potentially problematic PR for the following reasons:

• More than 50 changed files.
• More than 50 commits.

Such PR can only be merged by human. Contact a Core team member to review this PR on IRC: #ansible-devel on Libera.chat IRC

click here for bot help

[russoz]

hi @kosalaat it seems that you inadvertently merged a number of other commits into your PR. There are ways to undo that, but my git-fu does not go as far as doing that smoothly. When that happened to me before, I ended up creating a new PR.

For next occasions, use git rebase main instead. See https://github.com/ansible-collections/community.general/blob/main/CONTRIBUTING.md#open-pull-requests for more details.

[russoz]

Beware of #5097 - I don't see any immediate conflict but you and @aidanbh might want to coordinate so you do not to step on each other's toes.

[kosalaat]

Beware of #5097 - I don't see any immediate conflict but you and @aidanbh might want to coordinate so you do not to step on each other's toes.

@russoz I did comment on that PR. I do not see any issue with that, only few issues from terrform point of view.

[kosalaat]

If any of the git champs listening on this thread :), I'm desperately in need of git-fu help. How can I shake off the other PR's which is currently got tagged in with rebase?

[felixfontein]

I'm not sure exactly what happens, but usually what you do is:

1. Switch to the main branch of your local checkout
2. git pull upstream main, where upstream is the remote that points to this repository (and not your fork)
3. Then switch back to your branch
4. git rebase main - or if that has conflicts, git rebase --abort, and then git rebase -i main and remove everything from the list which isn't a commit by yourself.

[kosalaat]

Thanx @felixfontein for the pointer. Since I pushed the mess in, it was way pass the point I could recover gracefully. I think I fixed it with a hard reset to before my mistake.

1. git diff 30e6ed7f 21edebee --no-color > ~/Documents/terraform.py.final.patch
2. git reset --hard b33a34fe
3. git pull upstream main. This does a rebase to main
4. git apply ~/Documents/terraform.py.final.patch
5. do a stage and commit.
6. git push -f

Don't think the most graceful approach, but seem to have done the trick.

[ansibullbot]

The test licenses failed with 6 errors:

tests/integration/inventory:0:0: found no copyright notice
tests/integration/inventory:0:0: must have at least one license
tests/integration/targets/terraform/files/complex_variables/main.tf:0:0: found no copyright notice
tests/integration/targets/terraform/files/complex_variables/main.tf:0:0: must have at least one license
tests/integration/targets/terraform/files/complex_variables/variables.tf:0:0: found no copyright notice
tests/integration/targets/terraform/files/complex_variables/variables.tf:0:0: must have at least one license

The test ansible-test sanity --test yamllint [explain] failed with 2 errors:

tests/integration/targets/terraform/tasks/complex_variables.yml:57:1: empty-lines: too many blank lines (1 > 0)
tests/integration/targets/terraform/tasks/main.yml:68:1: empty-lines: too many blank lines (1 > 0)

The test ansible-test sanity --test yamllint [explain] failed with 2 errors:

tests/integration/targets/terraform/tasks/complex_variables.yml:57:1: empty-lines: too many blank lines (1 > 0)
tests/integration/targets/terraform/tasks/main.yml:68:1: empty-lines: too many blank lines (1 > 0)

The test ansible-test sanity --test yamllint [explain] failed with 2 errors:

tests/integration/targets/terraform/tasks/complex_variables.yml:57:1: empty-lines: too many blank lines (1 > 0)
tests/integration/targets/terraform/tasks/main.yml:68:1: empty-lines: too many blank lines (1 > 0)

The test ansible-test sanity --test yamllint [explain] failed with 2 errors:

tests/integration/targets/terraform/tasks/complex_variables.yml:57:1: empty-lines: too many blank lines (1 > 0)
tests/integration/targets/terraform/tasks/main.yml:68:1: empty-lines: too many blank lines (1 > 0)

click here for bot help

[ansibullbot]

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:0:0: doc-type-does-not-match-spec: Argument 'complex_vars' in argument_spec defines type as <class 'bool'> but documentation defines type as 'bool'

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:0:0: doc-type-does-not-match-spec: Argument 'complex_vars' in argument_spec defines type as <class 'bool'> but documentation defines type as 'bool'

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:0:0: doc-type-does-not-match-spec: Argument 'complex_vars' in argument_spec defines type as <class 'bool'> but documentation defines type as 'bool'

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:0:0: doc-type-does-not-match-spec: Argument 'complex_vars' in argument_spec defines type as <class 'bool'> but documentation defines type as 'bool'

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:94:161: E501: line too long (269 > 160 characters)

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:94:161: E501: line too long (269 > 160 characters)

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:94:161: E501: line too long (269 > 160 characters)

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:94:161: E501: line too long (269 > 160 characters)

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:94:161: E501: line too long (269 > 160 characters)

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

plugins/modules/cloud/misc/terraform.py:0:0: doc-type-does-not-match-spec: Argument 'complex_vars' in argument_spec defines type as <class 'bool'> but documentation defines type as 'bool'

click here for bot help

[kosalaat]

Hi @felixfontein, I agree with your concern. Considering how different strings are handled in the new code and the old code, the new interpretations would likely to break the existing use cases.

Added complex_vars => bool which defaults to false to makes the existing code to work without any changes unless complex_vars=true which reverts to the new code. This will make sure existing code/use cases continue to operate as is, but allows to switch to the new code if requested.

[patchback[bot]]

Backport to stable-5: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-5/beef93f687b9255ec675f078b9e93e78b938a81e/pr-4797

Backported as https://github.com/ansible-collections/community.general/pull/5331

🤖 @patchback I'm built with octomachinery and my source is open — https://github.com/sanitizers/patchback-github-app.

[felixfontein]

@kosalaat thanks a lot for your contribution! @russoz thanks for reviewing!

[kosalaat]

Thanx a lot @felixfontein / @russoz for all the guidance and patience. Much appreciated.