Data source "external" fails on Linux platform

[YuriGal]

Terraform CLI and Provider Versions

Terraform CLI: 1.1.1

External provider: 2.2.2

Terraform Configuration

data "external" "tools_accounts" {
   program  = ["aws", "organizations", "list-accounts-for-parent", "--parent-id", local.deployment_ou, "--query", "Accounts[?Name==`${local.tools_account}`] | [0]"]
}

Expected Behavior

When that external AWS CLI command runs it returns result similar to below:

{
    "Id": "1234567890",
    "Arn": "arn:aws:organizations::1234567890:account/o-p12345e/1234567890",
    "Email": "[email protected]",
    "Name": "account-alias",
    "Status": "ACTIVE",
    "JoinedMethod": "CREATED",
    "JoinedTimestamp": "2022-06-15T14:00:22.158000-04:00"
}

External command returns map of string values which satisfies the data source requirement. Data source's result output should provide this map.

Actual Behavior

When running inside of AWS Codebuild/Linux environment it throws the error

Cannot unmarshal number into Go value of type string

Steps to Reproduce

1. terraform apply

How much impact is this issue causing?

Medium

Logs

https://gist.github.com/YuriGal/fc74a9d87d3ab3148af58b569531a15d

Additional Information

The above configuration works on local laptop (Mac/Darwin, same terraform version, same provider version). I it just fails in AWS Codebuild/Linux environment.

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[YuriGal]

Is there a way to see raw command being executed and raw data being returned?

[bflad]

Hi @YuriGal 👋 Thank you for raising this and sorry you ran into trouble here.

Recent versions of the external provider should include a trace level log entry of the command if Terraform is ran with trace logging enabled, e.g.

terraform {
  required_providers {
    external = {
      source  = "hashicorp/external"
      version = "2.2.2"
    }
  }
}

data "external" "test" {
  program = ["echo", "-n", "{}"]
}

TF_LOG=TRACE terraform apply
...
2022-10-17T13:12:43.283-0400 [TRACE] provider.terraform-provider-external_v2.2.2_x5: Executing external program: @caller=github.com/terraform-providers/terraform-provider-external/internal/provider/data_source.go:154 @module=provider tf_data_source_type=external tf_provider_addr=provider tf_rpc=ReadDataSource program="/bin/echo -n {}" tf_req_id=c1a276d7-320f-addb-140a-f219f9e04e6a timestamp=2022-10-17T13:12:43.283-0400
2022-10-17T13:12:43.290-0400 [TRACE] provider.terraform-provider-external_v2.2.2_x5: Executed external program: program="/bin/echo -n {}" tf_rpc=ReadDataSource @caller=github.com/terraform-providers/terraform-provider-external/internal/provider/data_source.go:158 @module=provider output="{}" tf_data_source_type=external tf_provider_addr=provider tf_req_id=c1a276d7-320f-addb-140a-f219f9e04e6a timestamp=2022-10-17T13:12:43.290-0400
...

In particular, these structured logging fields: program="/bin/echo -n {}" and output="{}".

I'm guessing that AWS may be returning either the Id field as a JSON number or adding additional fields in that environment that happen to contain JSON numbers. We have an open issue for supporting dynamic output types in #76.

[bflad]

Without additional feedback, I'm going to guess this is covered by the feature request in #76. If I am mistaken, please let me know.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.