terraform-plugin-framework icon indicating copy to clipboard operation
terraform-plugin-framework copied to clipboard

Fix truncated diagnostics in go-cmp diffs in tests

Open paddycarver opened this issue 4 years ago • 2 comments

We use go-cmp pretty extensively in our tests to compare values. This works great, except when comparing diagnostics. As documented in google/go-cmp#279, our diagnostic detail output usually gets truncated, mostly right after telling us that an error occurred and it's always a problem with the provider. This leads to us littering logging statements around logging error output in our tests, which feels suboptimal.

I think we can write a customer Reporter but that seems like a large-ish undertaking, requires us to pass an option to all our calls to cmp.Diff, and I'm confused about whether we can use that for Diff or if it's just for Equal.

Seeing how google/go-cmp#279 resolves may help guide what we want to do here.

paddycarver avatar Oct 28 '21 22:10 paddycarver

Now tracking upstream at https://github.com/google/go-cmp/issues/264

bflad avatar Dec 13 '21 17:12 bflad

A unit testing workaround is looping through diagnostics with t.Logf() with something like:

if diff := cmp.Diff(resp.Diagnostics, testCase.expectedDiagnostics); diff != "" {
  for _, diagnostic := range resp.Diagnostics {
    t.Logf("unexpected diagnostic: %s", diagnostic)
  }
  t.Fatalf("unexpected diagnostics difference: %s", diff)
}

While we could say this is the desired implementation until there is further clarity on go-cmp changes or recommendations, this is hard to "enforce" so it is generally used as a temporary unit test troubleshooting step, rather than inconsistently implemented across the various unit testing.

bflad avatar Feb 07 '22 17:02 bflad

We're still waiting on https://github.com/google/go-cmp/issues/264 upstream for this and trying to pare down the issue tracker to actionable items, for reasons. Let's stay subscribed upstream and use the t.Logf() workaround for now until someone has some time to investigate further.

bflad avatar Sep 23 '22 16:09 bflad

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

github-actions[bot] avatar Oct 24 '22 02:10 github-actions[bot]