Error while scanning column of type `array<decimal(10,2)>` into Go type `[]float64`

[esdrasbeleza]

I have a table that has a column AMOUNTS with a type array<decimal(10,2)>, which I expect to scan into a field type []float64. This is the test I wrote to reproduce it:

func (s *DatabricksRepositorySuite) Test_Float64_Conversion() {
	query := `SELECT AMOUNTS FROM components WHERE AMOUNTS IS NOT NULL AND array_size(AMOUNTS) > 0 LIMIT 1`
	var result []float64
	err := s.repository.db.QueryRow(query).Scan(&result)
	s.NoError(err)
}

When I run my query, I get this error:

sql: Scan error on column index 0, name "AMOUNTS": unsupported Scan, storing driver.Value type string into type *[]float64

Source

This error comes from the standard library in Go: the value for that field is provided by the SDK as a string, but the destination is []float64, so it fails. I also dove into Databricks' SDK source code and found the place [2] where the string is generated.

[1] Golang source - convertAssignRows [2] Databricks SDK - listValueContainer.Value

Workaround

I created a custom type for the fields I have that use arrays:

import (
	"database/sql/driver"
	"encoding/json"
	"errors"
)

type databricksArray[T any] struct {
	parsedValue []T
}

func (f *databricksArray[T]) Scan(value interface{}) error {
	switch typedValue := value.(type) {
	case []T:
		f.parsedValue = typedValue
		return nil
	case string:
		return f.parseString(typedValue)
	default:
		return errors.New("error while handling databricks type")
	}
}

func (f *databricksArray[T]) parseString(value string) error {
	// This is a workaround for Databricks returning float arrays as strings
	// Example: `[1.0, 2.0, 3.0]`
	// We need to convert it to an array
	// This should be removed if/when Databricks fixes this issue
	valueBytes := []byte(value)
	var parsedValue []T
	if err := json.Unmarshal(valueBytes, &parsedValue); err != nil {
		return err
	}
	f.parsedValue = parsedValue
	return nil
}

func (f *databricksArray[T]) Value() (driver.Value, error) {
	return f.parsedValue, nil
}

func (f *databricksArray[T]) Slice() []T {
	if f == nil {
		return nil
	}
	return f.parsedValue
}

So my field

Amount []float64 `db:"AMOUNTS"`

is now

Amount databricksArray[float64] `db:"AMOUNTS"`

and I call Amount.Slice() when I want to read its value.