List ordering isn't stable/consistent between plans/state

[bitemyapp]

incidentally I was using HEAD of this repository in the course of debugging a very confusing issue with a non-null int64 value getting nulled out on the way through parsing and validation during terraform plan. It looked fine in the tfstate too. Now I've bumped into an issue with lists not being stable on HEAD. I didn't see this with v0.5.0 I think.

Now if you tell me that the list type value has be made stable by the provider that's fine but I'd like to know why it worked fine before but doesn't now. I'm going to look into the ordered set type now too.

Update: Never happened before tonight but now it's happening with these versions too:

	github.com/hashicorp/terraform-plugin-framework v0.5.0
	github.com/hashicorp/terraform-plugin-go v0.6.0

Update 2: I added stable sorting to the parent dataset list in the API and in the Provider and I'm still getting this error.

Module version

	github.com/hashicorp/terraform-plugin-framework v0.5.1-0.20220128221936-5dba37d74c08
	github.com/hashicorp/terraform-plugin-go v0.7.0
	github.com/hashicorp/terraform-plugin-sdk/v2 v2.10.1

Relevant provider source code

Some elision, but tried to isolate it to the relevant part.

...
"parent_datasets": {
	Required:   true,
	Attributes: tfsdk.ListNestedAttributes(schemaCoriariaParentDatasets("parent_datasets."), tfsdk.ListNestedAttributesOptions{}),
},
...

func schemaCoriariaParentDatasets(parent string) map[string]tfsdk.Attribute {
	return map[string]tfsdk.Attribute{
		"dataset_name": {
			Type:     types.StringType,
			Required: true,
		},
		"dependency_link": {
			Required:   true,
			Attributes: tfsdk.SingleNestedAttributes(schemaCoriariaDependencyLink(parent + "dependency_link.")),
		},
	}
}

func schemaCoriariaDependencyLink(parent string) map[string]tfsdk.Attribute {
	return map[string]tfsdk.Attribute{
		"unitary": {
			Type:     types.BoolType,
			Optional: true,
			// ConflictsWith: []string{parent + "identical", parent + "unbounded_upstream", parent + "bounded_upstream"},
		},
		"identical": {
			Type:     types.BoolType,
			Optional: true,
			// ConflictsWith: []string{parent + "unitary", parent + "unbounded_upstream", parent + "bounded_upstream"},
		},
		"unbounded_upstream": {
			Type:     types.BoolType,
			Optional: true,
			// ConflictsWith: []string{parent + "unitary", parent + "identical", parent + "bounded_upstream"},
		},
		"bounded_upstream": {
			Optional:   true,
			Attributes: tfsdk.SingleNestedAttributes(schemaCoriariaBoundedUpstream(parent + "bounded_upstream.")),
		},
	}
}

Terraform Configuration Files

resource "coriaria_dataset" "conversion_account_verification_event" {
  dataset_name = "conversion_account_verification_event"
  ...
  parent_datasets = [
    {
        dataset_name = coriaria_dataset.tbladvertiser_as_of.dataset_name
        dependency_link = {
            identical = true
        }
    },
    {
        dataset_name = coriaria_dataset.tblaccount_metadata_history.dataset_name
        dependency_link = {
            unbounded_upstream = true
        }
    }
  ]
  validations = []
}

Debug Output

2022-01-29T18:48:39.351-0600 [DEBUG] provider.terraform-provider-coriaria: 2022/01/29 18:48:39 [DEBUG] request JSON was: {"backend":{"backend_check_strategy":"once","catalog_id":"998954852051","database_name":"smaq_qa","s3_data_availability_type":"batch","slice_column":null,"table_name":"conversion_account_verification_event"},"builder_kind":{"athena_builder":{"bucket_name":"smb_measurements_qa","compression":"snappy","database_name":"smaq_qa","key":"coriaria/conversion_account_verification_event/","query":"conversion_account_verification_event.athena","query_argument_type":"daily_incremental","serde":"orc","table_name":"conversion_account_verification_event","temp_bucket":"smb-measurements-tmp","temp_database":"tmp"}},"dataset_id":null,"dataset_name":"conversion_account_verification_event","parent_datasets":[{"dataset_name":"tbladvertiser_as_of","dependency_link":"identical"},{"dataset_name":"tblaccount_metadata_history","dependency_link":"unbounded_upstream"}],"slice_range":{"temporal":{"end":{"fixed":"2022-01-07T00:00:00"},"interval":{"interval_quantity":1,"interval_type":"day"},"start":{"fixed":"2022-01-01T00:00:00"}}},"validations":[]}
2022-01-29T18:48:39.351-0600 [DEBUG] provider.terraform-provider-coriaria: 2022/01/29 18:48:39 [TRACE] Coriaria client had no error
2022-01-29T18:48:39.351-0600 [DEBUG] provider.terraform-provider-coriaria: 2022/01/29 18:48:39 [TRACE] Coriaria client had no error
2022-01-29T18:48:39.434-0600 [DEBUG] provider.terraform-provider-coriaria: 2022/01/29 18:48:39 [DEBUG] response Bytes were: {"dataset_id":"84","dataset_name":"conversion_account_verification_event","builder_kind":{"athena_builder":{"query":"conversion_account_verification_event.athena","query_argument_type":"daily_incremental","database_name":"smaq_qa","table_name":"conversion_account_verification_event","bucket_name":"smb_measurements_qa","key":"coriaria/conversion_account_verification_event/","compression":"snappy","serde":"orc","temp_database":"tmp","temp_bucket":"smb-measurements-tmp"}},"slice_range":{"temporal":{"start":{"fixed":"2022-01-01T00:00:00"},"end":{"fixed":"2022-01-07T00:00:00"},"interval":{"interval_type":"day","interval_quantity":1}}},"backend":{"catalog_id":"998954852051","database_name":"smaq_qa","table_name":"conversion_account_verification_event","slice_column":null,"s3_data_availability_type":"batch","backend_check_strategy":"once"},"parent_datasets":[{"dataset_name":"tblaccount_metadata_history","dependency_link":"unbounded_upstream"},{"dataset_name":"tbladvertiser_as_of","dependency_link":"identical"}],"validations":[]}

Take-away on the debug output is that the API responded with a differently ordered list:

Expected Behavior

The plan should've been clean and matched the state after apply.

Actual Behavior

╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to coriaria_dataset.conversion_account_verification_event, provider
│ "provider[\"indeed.com/indeed/coriaria\"]" produced an unexpected new value:
│ .parent_datasets[0].dependency_link.identical: was cty.True, but now null.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to coriaria_dataset.conversion_account_verification_event, provider
│ "provider[\"indeed.com/indeed/coriaria\"]" produced an unexpected new value:
│ .parent_datasets[0].dependency_link.unbounded_upstream: was null, but now cty.True.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to coriaria_dataset.conversion_account_verification_event, provider
│ "provider[\"indeed.com/indeed/coriaria\"]" produced an unexpected new value: .parent_datasets[0].dataset_name:
│ was cty.StringVal("tbladvertiser_as_of"), but now cty.StringVal("tblaccount_metadata_history").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to coriaria_dataset.conversion_account_verification_event, provider
│ "provider[\"indeed.com/indeed/coriaria\"]" produced an unexpected new value:
│ .parent_datasets[1].dependency_link.identical: was null, but now cty.True.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵

Steps to Reproduce

I think you could repro it with any list type.