avro icon indicating copy to clipboard operation
avro copied to clipboard

panic when encoding array that can contain null values

Open jgresty opened this issue 2 years ago • 1 comments

When trying to encode the data type:

type Data struct {
        Data *[]*string `json:"data"`
}

which is generated with avrogo from the schema

{
  "type": "record",
  "name": "Data",
  "namespace": "test",
  "connect.version": 1,
  "connect.name": "test.Data",
  "default": null,
  "fields": [
    {
      "name": "data",
      "default": null,
      "type": [
        "null",
        {
          "type": "array",
          "items": ["null", "string"]
        }
      ]
    }
  ]
}

results in the error:

panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
github.com/heetch/avro.(*encoderBuilder).typeEncoder(0xc000137cc8, {0x539498?, 0xc00012e190?}, {0x53a0a0, 0x4eb000}, {{0x53a0a0, 0x4eb700}, {0x0, 0x0}, 0x0, ...})
	/home/jgresty/go/pkg/mod/github.com/heetch/[email protected]/encode.go:173 +0x190f
github.com/heetch/avro.(*encoderBuilder).typeEncoder(0xc000137cc8, {0x538e08?, 0xc0001282b8?}, {0x53a0a0, 0x4eb700}, {{0x53a0a0, 0x4eb700}, {0x0, 0x0}, 0x0, ...})
	/home/jgresty/go/pkg/mod/github.com/heetch/[email protected]/encode.go:208 +0xbd6
github.com/heetch/avro.(*encoderBuilder).typeEncoder(0xc000137cc8, {0x539498?, 0xc00012e1e0?}, {0x53a0a0, 0x4e8fc0}, {{0x53a0a0, 0x4e8fc0}, {0x4e63e1, 0x4}, 0x0, ...})
	/home/jgresty/go/pkg/mod/github.com/heetch/[email protected]/encode.go:176 +0x3c6
github.com/heetch/avro.(*encoderBuilder).typeEncoder(0xc000177cc8, {0x5393b8?, 0xc00011ca20?}, {0x53a0a0, 0x4f8640}, {{0x53a0a0, 0x4f8640}, {0x0, 0x0}, 0x0, ...})
	/home/jgresty/go/pkg/mod/github.com/heetch/[email protected]/encode.go:149 +0x15b6
github.com/heetch/avro.typeEncoder(0xc00012c120, {0x53a0a0?, 0x4f8640})
	/home/jgresty/go/pkg/mod/github.com/heetch/[email protected]/encode.go:71 +0x210
github.com/heetch/avro.marshalAppend(0xc000164100?, {0xc00016e000, 0x1, 0x64}, {0x4f8640?, 0x0?, 0x0?})
	/home/jgresty/go/pkg/mod/github.com/heetch/[email protected]/encode.go:38 +0xd3
github.com/heetch/avro.(*SingleEncoder).Marshal(0xc000164100, {0x538850, 0xc000120000}, {0x4f8640?, 0x0?})
	/home/jgresty/go/pkg/mod/github.com/heetch/[email protected]/singleencoder.go:64 +0x1ad
main.main()

I have a runnable example at: https://github.com/jgresty/avrotest

jgresty avatar Dec 13 '22 12:12 jgresty

Hey @jgresty thanks a lot for opening that issue and especially for the code to reproduce this, and apologies for the delay in responding to this. Unfortunately we do not have much bandwidth currently to work on this.

If you have control over the schema, I would suggest maybe wrapping the value in a record that has a nullable field?

{
  "type": "record",
  "name": "Data",
  "namespace": "test",
  "connect.version": 1,
  "connect.name": "test.Data",
  "default": null,
  "fields": [
    {
      "name": "data",
      "default": null,
      "type": [
        "null",
        {
          "type": "array",
          "items":{
                "name":"ArrayValue",
                "type":"record",
                "fields":[
                    {"name":"value", "type":["null", "string"]}
                ]
            }
        }
      ]
    }
  ]
}

skateinmars avatar Jan 25 '23 17:01 skateinmars