Provider produced inconsistent final plan

[mkeeler]

I have a module which uses dynamic blocks for the ports networks_advanced and upload fields of a docker_container resource. This module at various times will cause terraform to emit errors about the docker provider producing an inconsistent final plan which said that it is a provider bug and to open the issue here.

I am not so certain this is a bug in the docker provider but rather have a hunch that the actual problem is in terraform relating to dynamic blocks. Regardless I am opening this issue to start the process of figuring out why this would be happening.

Terraform Version

Terraform Version: v0.12.21 Provider Version: v2.7.0

Affected Resource(s)

• docker_container

Terraform Configuration Files

This is weird to reproduce and involves a module and some tf code so I setup a repo with all the files:

https://github.com/mkeeler/tf-issue-repro

The modules directory there contains the module I am using which is where its using the docker_container resource. The repro directory contains the main terraform code that I plan/apply.

Debug Output

Output from running terraform apply: https://gist.github.com/mkeeler/d79a9748338019e46a3d3d35dc48b6b2

Output with TF_LOG=TRACE: https://gist.github.com/mkeeler/d0d485d2df6d1084eadc125d41825482

Expected Behavior

No errors should occur.

Actual Behavior

While the resources were created properly terraform emitted the following errors:

Error: Provider produced inconsistent final plan

When expanding the plan for
module.clients.docker_container.client-containers[0] to include new values
learned so far during apply, provider "registry.terraform.io/-/docker"
produced an invalid new value for .ports: block count changed from 1 to 2.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.


Error: Provider produced inconsistent final plan

When expanding the plan for
module.clients.docker_container.client-containers[0] to include new values
learned so far during apply, provider "registry.terraform.io/-/docker"
produced an invalid new value for .upload: planned set element
cty.ObjectVal(map[string]cty.Value{"content":cty.UnknownVal(cty.String),
"content_base64":cty.StringVal(""), "executable":cty.False,
"file":cty.UnknownVal(cty.String), "source":cty.StringVal(""),
"source_hash":cty.StringVal("")}) does not correlate with any element in
actual.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply (to see the odditites of the plan)
2. Allow the apply to happen by typing yes to get the errors.

Important Factoids

• The trace output shows many more warnings about the provider producing invalid plans but terraform allowing it. These look like:

2020/03/30 09:52:28 [WARN] Provider "registry.terraform.io/-/docker" produced an invalid plan for module.clients.docker_container.client-containers[0], but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .must_run: planned value cty.True does not match config value cty.NullVal(cty.Bool)
      - .log_driver: planned value cty.StringVal("json-file") does not match config value cty.NullVal(cty.String)
      <lots more repeats of issues with field types here>```

• The plan that terraform produces only includes a single ports block even though I have two ports being configured. The provider still seems to correctly port map both ports as desired so I suspect the root problem is something with the planning process. My tf code sets up some inputs to my module to configure 2 ports here. Then the module sets up the dynamic ports block in the docker_container resource here.

• Not sure why but the original single ports block says that all the fields besides the IP would be known after apply. While all the data is coming through module inputs/local vars there is nothing dynamic in nature about the source of the external, internal and protocol fields.

        + ports {
          + external = (known after apply)
          + internal = (known after apply)
          + ip       = "0.0.0.0"
          + protocol = (known after apply)
        }

[mavogel]

Hey, thanks for pointing this one out and I have a feeling that it's most probably what is planned to be fixed on #138 . There you can see the history of the issue. Do you have any idea how to implement this properly? BC is fine for v3.0.0

[mkeeler]

It might be related but if so it also affects other fields such as the "upload":

- Provider produced inconsistent final plan: When expanding the plan for module.clients.docker_container.client-containers[0] to include new values learned so far during apply, provider "registry.terraform.io/-/docker" produced an invalid new value for .upload: planned set element cty.ObjectVal(map[string]cty.Value{"content":cty.UnknownVal(cty.String), "content_base64":cty.StringVal(""), "executable":cty.False, "file":cty.UnknownVal(cty.String), "source":cty.StringVal(""), "source_hash":cty.StringVal("")}) does not correlate with any element in actual.```

Given that it seems to happen with both uploads and ports my initial thought was that it was somehow a problem with dynamic blocks in terraform itself.

As for #138 and implementation ideas, I don't personally have much need for randomly assigned ports but do see why it could be generally useful. Also I can see why the change from a set to a list was necessary to support that but don't really know how to implement it properly. Was the original problem that the hashing was ignoring computed fields or something causing fewer elements in the set than there should be? I tried looking at the related issues and PRs but it wasn't immediately clear why using the set wasn't working.

[GMSteuart]

Without digging into the codebase, I imagine there is an issue with determining the diff between user defined changes and docker cli changes, or some expectation that docker containers and services will all have the same fields defined. As I have gotten this error with placement and its children constraints and prefs.