state

[automaticgiant]

i had some thoughts/concerns about the state object.

initially, i was confused. i didn't understand why there was a state object, whose internal dict was replaced with the parsed json. was thinking about ignoring and/or contributing/suggesting changing it to make the normal nested dict available (instead of using tf.modules[0]... would like to do it as if i were using the state json externally.

then i saw https://github.com/beelit94/python-terraform/issues/11 , and the work going into state in https://github.com/beelit94/python-terraform/pull/19/files , and it makes more sense.

but i'm slightly conflicted. https://github.com/UCL-CloudLabs/deployer/issues/14#issuecomment-381202739 and remote state are good reason for the state abstraction, but would still love to be able to get the json(nested dicts) back instead (or additionally). want to be able to write code that operates on the same data structure i have from the state file itself, ideally.

thoughts @beelit94 ? thanks hunter

[beelit94]

@automaticgiant for state file, I find it hard to understand the structure as well, so I always define "output" to fetch the information I want. for how to get a better state structure, I believe https://github.com/rodynnz/python-tfstate repo also work on this, maybe we can bring the library in? I think it would be more helpful if you can describe a use case of you so we can move this forward :)

[automaticgiant]

example usage: (fixture provides an applied tf config/state and destroys after)

def test_hit_bwaf(module):
    attributes = module.tfstate.modules[0]['resources']['aws_route53_record.internal-w2']['primary']['attributes']
    fqdn = attributes['fqdn']
    r = requests.get('https://'+fqdn, verify=False)
    assert '0123456789'*50 in r.text

in python-tfstate, i see the native_data attribute, which i like. right now i'm thinking that i at least want to support that, while the rest of the abstraction is potentially in flux. then we won't have to rewrite our existing tests. other than that, can't say i really grok python-tfstate yet - maybe could provide more input after there are docs for it.

basically, i want to avoid the (currently underdeveloped, and subject to change) state api in this library for now. i see lots of potential, but perhaps this library should avoid it all and just give a state file reference. the improvements we're talking about in this repo are potentially better suited to the other lib too. then ppl can use the tfstate lib if they want, or just parse/deal with the default json. that also means this library won't be dependent on tfstate schema (changes).

thoughts?

[beelit94]

Hi @automaticgiant , I also implemented, native_data in our tfstate object, https://github.com/beelit94/python-terraform/blob/develop/python_terraform/tfstate.py I guess this should be enough for the use case for now?

[Spikeophant]

I thought it might be worthwhile to note here for remote state with the release of Terraform 0.12 we'll be able to retrieve the remote state in JSON format, making it so there will just need to be a check to see if there's a backend config, and if so run terraform show -json to get the output rather than read in from a file. I was getting ready to do the work to read the state in this way when I realized it was limited to 0.12 and I am not quite ready to start working with that version since it's not production ready. I need to read certain elements from state and pass them to other processes running in my CI pipelines, so would be happy to tackle adding remote state support once Terraform 0.12 is officially released, or possibly before if I can get time outside of work to mess with this.