Broken reference completion w/ trailing dot inside complex types `{}` / `[]`

[radeksimko]

Context

Reference completion often includes trailing dot, which is the step separator. The trailing dot is not included in the AST by HCL, which we already account for in the obvious cases:

https://github.com/hashicorp/hcl-lang/blob/bd545f4d6346360fb9a4e2e4a81423fa212c581e/decoder/expr_reference_completion.go#L71-L82

However we do not seem to account for similar cases involving references with trailing dot inside complex types (list, set, map, maybe object as well):

Debugging Details

For list specifically I was able to narrow down the codepath to this condition, which compares the element range with the given position, which is not contained (it's off by one due to the dot that isn't included in the range):

https://github.com/hashicorp/hcl-lang/blob/bd545f4d6346360fb9a4e2e4a81423fa212c581e/decoder/expr_list_completion.go#L77-L79

It is likely that other complex types have the same root cause.

Proposal

Fix the bug by providing completion of nested references after the trailing dot.

Ideas

We could leverage the recovery mechanism to recover the whole expression:

https://github.com/hashicorp/hcl-lang/blob/bd545f4d6346360fb9a4e2e4a81423fa212c581e/decoder/expression.go#L226-L253

The only extra complexity is that we'd have to pass around those recovered bytes (maybe as prefix?) in addition to the AST (hcl.Expression) between all expression types.

[dbanck]

Here is a test case for objects

func TestCompletionAtPos_exprObject_references(t *testing.T) {
	testCases := []struct {
		testName           string
		attrSchema         map[string]*schema.AttributeSchema
		refTargets         reference.Targets
		cfg                string
		pos                hcl.Pos
		expectedCandidates lang.Candidates
	}{
		{
			"single-line element with trailing dot",
			map[string]*schema.AttributeSchema{
				"attr": {
					Constraint: schema.Object{
						Attributes: schema.ObjectAttributes{
							"foo": {
								IsOptional: true,
								Constraint: schema.Reference{OfScopeId: lang.ScopeId("variable")},
							},
						},
					},
				},
			},
			reference.Targets{
				{
					Addr: lang.Address{
						lang.RootStep{Name: "var"},
						lang.AttrStep{Name: "bar"},
					},
					RangePtr: &hcl.Range{
						Filename: "variables.tf",
						Start:    hcl.Pos{Line: 2, Column: 1, Byte: 17},
						End:      hcl.Pos{Line: 2, Column: 3, Byte: 19},
					},
					ScopeId: lang.ScopeId("variable"),
				},
			},
			`attr = { foo = var. }
`,
			hcl.Pos{Line: 1, Column: 20, Byte: 19},
			lang.CompleteCandidates([]lang.Candidate{
				{
					Label:  "var.bar",
					Detail: "reference",
					Kind:   lang.TraversalCandidateKind,
					TextEdit: lang.TextEdit{
						NewText: "var.bar",
						Snippet: "var.bar",
						Range: hcl.Range{
							Filename: "test.tf",
							Start:    hcl.Pos{Line: 1, Column: 16, Byte: 15},
							End:      hcl.Pos{Line: 1, Column: 20, Byte: 19},
						},
					},
				},
			}),
		},
	}

	for i, tc := range testCases {
		t.Run(fmt.Sprintf("%d-%s", i, tc.testName), func(t *testing.T) {
			bodySchema := &schema.BodySchema{
				Attributes: tc.attrSchema,
			}

			f, _ := hclsyntax.ParseConfig([]byte(tc.cfg), "test.tf", hcl.InitialPos)
			d := testPathDecoder(t, &PathContext{
				Schema: bodySchema,
				Files: map[string]*hcl.File{
					"test.tf": f,
				},
				ReferenceTargets: tc.refTargets,
			})

			ctx := context.Background()
			candidates, err := d.CandidatesAtPos(ctx, "test.tf", tc.pos)
			if err != nil {
				t.Fatal(err)
			}

			if diff := cmp.Diff(tc.expectedCandidates, candidates); diff != "" {
				t.Logf("position: %#v in config: %s", tc.pos, tc.cfg)
				t.Fatalf("unexpected candidates: %s", diff)
			}
		})
	}
}