Better error messaging for "unhandled {null,unknown} value"

[kmoe]

When trying to reflect a state/plan value into a struct with plain Go types using Get(), the framework will throw an error if the value contains a Null or Unknown value. Developers receive a message saying "unhandled null value" or "unhandled unknown value" with little context.

Provider code

Example is from the framework-provider in terraform-provider-corner.

func (r resourceUserType) GetSchema(_ context.Context) (tfsdk.Schema, diag.Diagnostics) {
	return tfsdk.Schema{
		Attributes: map[string]tfsdk.Attribute{
			"name": {
				Type:     types.StringType,
				Required: true,
			},
			"id": {
				Type:     types.StringType,
				Computed: true,
			},
		},
	}, nil
}

type user struct {
	Name  string `tfsdk:"name"`
	Id    string `tfsdk:"id"`
}

func (r resourceUser) Create(ctx context.Context, req tfsdk.CreateResourceRequest, resp *tfsdk.CreateResourceResponse) {
	var plan user
	diags := req.Plan.Get(ctx, &plan)
	resp.Diagnostics.Append(diags...)
	if resp.Diagnostics.HasError() {
		return
	}

...

Config

resource "framework_user" "foo" {
  name = "Ford Prefect"
}

Output

When running terraform apply as part of an acc test:

--- FAIL: TestAccResourceUser (11.89s)
    resource_user_test.go:13: Step 1/1 error: Error running apply: exit status 1
        
        Error: Value Conversion Error
        
          with framework_user.foo,
          on terraform_plugin_test.tf line 2, in resource "framework_user" "foo":
           2: resource "framework_user" "foo" {
        
        An unexpected error was encountered trying to build a value. This is always
        an error in the provider. Please report the following to the provider
        developer:
        
        unhandled unknown value

Similarly, if the "id" attribute is instead marked as Optional: true and not set in the config, an "unhandled null value" error is thrown.

Conclusion

Consider logging the actual value that could not be converted.

Also, we may want to consider somehow extending the UnhandledNullAsEmpty and UnhandledUnknownAsEmpty reflect options into Get(), or perhaps an alternative "simple get" with these options set by default.

[paddycarver]

The last paragraph there I think is tracked as #84, but I do think we could have better error messages here, too.

[bflad]

Ha, I just ran into this myself while working on the scaffolding repository. It is definitely easy to get this wrong.

[binoy14]

Hi, I am building a new provider with terraform-plugin-framework. I keep running into this issue when I have an attribute as Optional and Computed. Is there a workaround for this or any more details on how to debug this? Appreciate any help!

"processing_settings": {
	Computed: true,
	Optional: true,
	Attributes: tfsdk.SingleNestedAttributes(map[string]tfsdk.Attribute{
		"skip": {
			Type:     types.BoolType,
			Optional: true,
			Computed: true,
		},
	}),

Versions

Terraform v1.1.2
github.com/hashicorp/terraform-plugin-framework v0.5.0

[paddycarver]

The struct you're passing a pointer to in Get should have processing_settings as a types.Object and skip as a types.Bool.

[apparentlymart]

An extra constraint on tfsdk.Plan and tfsdk.Config that doesn't apply to tfsdk.State is that the "known-ness" of any particular argument is under the control of the module author, not the provider developer. Any attribute could show up with an unknown value, and so therefore I don't think it's ever valid to decode from those data structures into a type that can't handle an unknown value.

Given that, would it be reasonable to change the signatures of Plan.Get and Config.Get to require a more specific interface type than interface{}, so that only custom types are allowed and it'd be a compile-time error to use a "normal" Go type in those contexts?

One immediate caveat that comes to my mind just from suggesting that is that it seems like the framework is reusing these two types across both the plan-time operations and the apply-time operations. The apply-time operations can safely assume that there won't be any unknown values remaining at that point, in which case it would be convenient to be able to use "normal" Go types.

Therefore I suppose in order to achieve what I described above without making the apply-time functions far less convenient it would require adding a tfsdk.PlanTimePlan and tfsdk.PlanTimeConfig which had the more specific signature, and making those be the ones that have the more specialized argument type, which is decidedly less clean than my original idea. :confounded:

Null values can appear in any phase though, so perhaps my focus on solving for unknown values here is unwarranted.

(Side note: I find myself wondering how the original example in the opening comment failed here, because it should in principle be impossible for Create to ever see an unknown value: Create presumably runs in response to ApplyResourceChange, and there shouldn't be any unknown values left by the time we're running that function. I wonder if there's something going wrong at a lower layer here in order for this to have arisen in the way it did, even though what I said above would still apply to e.g. a ModifyPlan method, which does need to deal with unknown values.)

[williansouzagonc]

That can be easily fixed using Modifiers.

	"id": {
		Type:     types.StringType,
		Computed: true,
		PlanModifiers: tfsdk.AttributePlanModifiers{
			tfsdk.UseStateForUnknown(),
		},
	}

[abergmeier]

tfsdk.UseStateForUnknown()

Can you elaborate, why it is necessary and what it actually does? It does not seem to help for my computed attrbiutes.

Currently get bitten by this in ValidateConfig and have no idea where this comes from. I am assuming that PlanModifiers will be applied when using req.Config.Get in ValidateConfig.

[iamdexterpark]

The struct you're passing a pointer to in Get should have processing_settings as a types.Object and skip as a types.Bool.

Deeply curious about this. I've yet to discover a v6 provider following the terraform plugin framework with concrete examples of how to implement tfsdk.Schema leveraging a list or hash map.

Personally I'm blocked on developing a terrific provider because of just two attributes causing unhandled null value error. It took me a while to realize these were the cause.

organizations.tf

resource "meraki_organization" "terraform1" {
  
  api = {
    enabled = true
  }
  
  licensing = {
    model = "example"
  }

}

GetSchema

return tfsdk.Schema{
MarkdownDescription: "Organization resource",
Attributes: map[string]tfsdk.Attribute{

                       // Omitting leading string attributes for brevity

                     // list containing single bool element
	"api": {   
		Optional: true,
		Attributes: tfsdk.SingleNestedAttributes(map[string]tfsdk.Attribute{
			"enabled": {
				Type:     types.BoolType,
				Optional: true,
			},
		}),
			},

                      // hash map containing single string element
	"licensing": {
		Computed: true,
		Optional: true,
		Attributes: tfsdk.ListNestedAttributes(map[string]tfsdk.Attribute{
			"model": {
				Type:     types.StringType,
				Optional: true,
				Computed: true,
			},
		}, tfsdk.ListNestedAttributesOptions{}),
	},
},
}

Structs

type OrganizationData struct {
	ID    types.String `tfsdk:"id"`
	Name  types.String `tfsdk:"name"`
	Url   types.String `tfsdk:"url"`
	Cloud types.String `tfsdk:"cloud"`
	API   Api    `tfsdk:"api"`
	Licensing Licensing    `tfsdk:"licensing"`
}

type Api struct {
	Enabled types.Bool `tfsdk:"enabled"`
}

type Licensing struct {
	Model types.String `tfsdk:"model"`
}

Read func

       // Map response body to resource schema attribute
        data.ID = types.String{Value: response.GetPayload().ID}
	data.Name = types.String{Value: response.GetPayload().Name}
	data.Url = types.String{Value: response.GetPayload().URL}
	data.Cloud = types.String{Value: response.GetPayload().Cloud.Region.Name}

	// Throwing unhandled null value error:
	data.Licensing.Model = types.String{Value: response.GetPayload().Licensing.Model}
	data.API.Enabled = types.Bool{Value: response.GetPayload().API.Enabled}

Commenting out the offending attributes causes the provider to run so it's just a matter of being unclear on how to implement a list or hash map in my resource.

Any guidance or documentation would be greatly appreciated. Thanks in advance.

[megan07]

@ddexterpark I've found that making Licensing a types.List in the struct was helpful for me, then as you're dealing with it you can use

licensing := Licensing{}
l.(types.Object).As(ctx, &licensing, types.ObjectAsOptions{})

to convert each element into the Licensing object.

That's assuming it's a list, if it's a single-nested object, then make it types.Object in the struct and use Attributes: tfsdk.SingleNestedAttributes in the schema. You would use that same conversion as above.

This is off the top of my head, so it might not be perfect. Also, I'm not done with my provider yet, so I might be missing pieces still too.

[iamdexterpark]

@ddexterpark I've found that making Licensing a types.List in the struct was helpful for me, then as you're dealing with it you can use

licensing := Licensing{}
l.(types.Object).As(ctx, &licensing, types.ObjectAsOptions{})

to convert each element into the Licensing object.

That's assuming it's a list, if it's a single-nested object, then make it types.Object in the struct and use Attributes: tfsdk.SingleNestedAttributes in the schema. You would use that same conversion as above.

This is off the top of my head, so it might not be perfect. Also, I'm not done with my provider yet, so I might be missing pieces still too.

@megan07 Wonderful, thank you for taking the time to look into this. I updated the original post to reflect the actually data structure in the .tf file for clarity on the subtypes.

Perhaps I'm misunderstanding something fundamental, may I ask where to declare types.Object? It's not clear to me if you are referring to the merakiOrganizationData or Licensing struct directly. Either case leaves me befuddled as to how licensing := Licensing{} asserts types.Object.

Alternatively I was able to get it working by declaring Licensing in the merakiOrganizationData struct as map[string]Licensing with a tfsdk.Schema entry set to:

"licensing": {
				Computed: true,
				Optional: true,
				Attributes: tfsdk.MapNestedAttributes(map[string]tfsdk.Attribute{
					"model": {
						Type:     types.StringType,
						Optional: true,
						Computed: true,
					},
				}, tfsdk.MapNestedAttributesOptions{}),
			},

and in my Read/Create/Update funcs:

var resourceResult = merakiOrganizationData{
		Licensing: map[string]Licensing{
			"model": {
				types.String{Value: response.GetPayload().Licensing.Model},
			},
		},
	}

diags = resp.State.Set(ctx, &resourceResult)

Same workflow with the api hash map but it doesn't feel as elegant a solution as yours. Could I trouble you for an extended code snippet of how you would implement this struct.

I'm trying to get as much of the solution on this thread as possible for the next developer who runs into this issue whom I'm sure will also deeply appreciate your help.

Thanks again!

[rschmied]

So I am running into a similar issue with the unknown value in Get() during ModifyPlan as indicated by @apparentlymart

Schema: (stripped down a bit for readability):

		Attributes: map[string]tfsdk.Attribute{
			"id": {
				Computed:            true,
				PlanModifiers: tfsdk.AttributePlanModifiers{
					tfsdk.UseStateForUnknown(),
				},
				Type: types.StringType,
			},
			"nodes": {
				Computed:            true,
				Attributes: tfsdk.ListNestedAttributes(
					nodeSchema(),
					tfsdk.ListNestedAttributesOptions{},
				),
				PlanModifiers: tfsdk.AttributePlanModifiers{
					tfsdk.UseStateForUnknown(),
				},
			},
		},

with NodeSchema like this:

func nodeSchema() map[string]tfsdk.Attribute {
	return map[string]tfsdk.Attribute{
		"id": {
			Type:                types.StringType,
			Computed:            true,
			PlanModifiers: tfsdk.AttributePlanModifiers{
				tfsdk.UseStateForUnknown(),
			},
		},
		"label": {
			Type:                types.StringType,
			Computed:            true,
			PlanModifiers: tfsdk.AttributePlanModifiers{
				tfsdk.UseStateForUnknown(),
			},
		},
      }
...

structs:

type cmlLabResourceData struct {
	Id       types.String   `tfsdk:"id"`
	Nodes    []types.Object `tfsdk:"nodes"`
}

and adding nodes to the list via this (wondering if there's a better way as this seems kind of ugly):

		o := types.Object{
			AttrTypes: map[string]attr.Type{
				"id":       types.StringType,
				"label":    types.StringType,
			},
			Attrs: map[string]attr.Value{
				"id":       types.String{Value: node.ID},
				"label":    types.String{Value: node.Label},
			},
		}
		nodes = append(nodes, o)

this works as long as I do not implement a ModifyPlan() func. As soon as there's no state, I get the Value Conversion Error with the helpful "unhandled unknown value".

I really would appreciate a more complete example of how to use lists of objects...

[multani]

I have a similar problem, but also a bit different.

I'm trying to implement a data source, for which I'd like to have a field which is a struct containing other fields:

package provider

import (
	"context"

	"github.com/hashicorp/terraform-plugin-framework/diag"
	"github.com/hashicorp/terraform-plugin-framework/tfsdk"
	"github.com/hashicorp/terraform-plugin-framework/types"
)

// Ensure provider defined types fully satisfy framework interfaces
var _ tfsdk.DataSourceType = myDataSourceType{}
var _ tfsdk.DataSource = myDataSource{}

type myDataSourceType struct{}

func (t myDataSourceType) GetSchema(ctx context.Context) (tfsdk.Schema, diag.Diagnostics) {
	return tfsdk.Schema{
		Attributes: map[string]tfsdk.Attribute{
			"id": {
				Type:     types.StringType,
				Computed: true,
			},

			"name": {
				Type:     types.StringType,
				Required: true,
			},

			"sub_field": {
				Computed: true,
				Attributes: tfsdk.SingleNestedAttributes(
					map[string]tfsdk.Attribute{
						"attr1": {
							Computed: true,
							Type:     types.StringType,
						},
						"attr2": {
							Computed: true,
							Type:     types.Int64Type,
						},
					},
				),
			},
		},
	}, nil
}

type myDataSource struct {
	provider provider
}

func (t myDataSourceType) NewDataSource(ctx context.Context, in tfsdk.Provider) (tfsdk.DataSource, diag.Diagnostics) {
	provider, diags := convertProviderType(in)

	return myDataSource{
		provider: provider,
	}, diags
}

type subField struct {
	Attr1 types.String    `tfsdk:"attr1"`
	Attr2 types.Int64Type `tfsdk:"attr2"`
}

type myDataSourceData struct {
	Id       types.String `tfsdk:"id"`
	Name     types.String `tfsdk:"name"`
	SubField subField     `tfsdk:"sub_field"`
}

func (d myDataSource) Read(ctx context.Context, req tfsdk.ReadDataSourceRequest, resp *tfsdk.ReadDataSourceResponse) {
	var data myDataSourceData

	diags := req.Config.Get(ctx, &data)
	resp.Diagnostics.Append(diags...)

	if resp.Diagnostics.HasError() {
		return
	}

	// TODO: fill in the data from the API
}

The goal would be to be able to use something like this:

data "my_data_source" "foo" {
  name = "foobar"
}

output "attr1" {
  value = data.my_data_source.foo.sub_field.attr1
}

In this case, there's no "plan" to get the content of the sub_field attribute from, but Terraform still fails with:

│ Error: Value Conversion Error
│ 
│   with data.my_data_source.foo,
│   on test.tf line 1, in data "my_data_source" "foo":
│   1: data "my_data_source" "foo" {
│ 
│ An unexpected error was encountered trying to build a value. This is always an error in the provider. Please report the following to the provider developer:
│ 
│ unhandled null value

Although the error message doesn't show it, this fails on the diags := req.Config.Get(ctx, &data) line, and I'm not sure what I can do about it, beside removing the structure and flattening the fields at the datasource level.

[pksunkara]

@bflad This seems to be a major issue and makes it impossible to work with non-required nested attributes. Is there any plan to fix this issue?

[bflad]

The next version of the framework will begin returning error diagnostics such as the following when null/unknown values cannot be handled when getting schema-based data:

Error: Value Conversion Error

  (configuration source pointing at attribute value, if it was configurable, refer also to https://github.com/hashicorp/terraform/issues/31575)

An unexpected error was encountered trying to build a value. This is always an error in the provider. Please report the following to the provider developer:

Received null value, however the target type cannot handle null values. Use the corresponding `types` package type, a pointer type or a custom type that handles null values.

Path: example_single_nested_attribute
Target Type: struct { NestedString types.String "tfsdk:\"nested_string\"" }
Suggested `types` Type: types.Object
Suggested Pointer Type: *struct { NestedString types.String "tfsdk:\"nested_string\"" }

Error: Value Conversion Error

  (configuration source pointing at attribute value, if it was configurable, refer also to https://github.com/hashicorp/terraform/issues/31575)

An unexpected error was encountered trying to build a value. This is always an error in the provider. Please report the following to the provider developer:

Received unknown value, however the target type cannot handle unknown values. Use the corresponding `types` package type or a custom type that handles unknown values.

Path: example_list_attribute
Target Type: []string
Suggested Type: types.List

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.