goavro icon indicating copy to clipboard operation
goavro copied to clipboard

Add `Codec.ScanBinary`

Open SpencerC opened this issue 1 year ago • 1 comments

Motivation

Currently, developers calling the Codec.Native functions need to write glue code to traverse the returned map[string]interface{}s and convert the values. While this approach makes sense for maintaining the symmetry between the encoder input and the decoder output and for its reflection of the underlying architecture of the format, it creates some unnecessary friction in applications where the goal is simply to get data into simple, native types.

Background

I recently created a Go SDK for DataStax Astra which provides methods for developers to work with returned data without worrying about the underlying API. It implements a Scan function that mirrors the functionality of the builtin database/sql#Rows.Scan function. I think a similar approach might be useful here.

convertAssign: https://github.com/datastax-ext/astra-go-sdk/pull/15 Row.Scan: https://github.com/datastax-ext/astra-go-sdk/pull/16

Future Work

  • Implement scan logic for maps and slices.
  • Create testBinaryScan function and add more test cases.
  • Create a Codec function to expose the field name order. Unlike the case where Scan calls follow database queries, schemas may be loaded from outside the code and the field order may not be visible to developers. Giving developers access to the field name order would allow them to create runtime mappings between fields and destinations (example)

SpencerC avatar Jun 07 '23 16:06 SpencerC

It turns out this approach is also more performant. I'm guessing because it requires fewer dynamic allocations? Benchmark results:

goos: darwin
goarch: arm64
pkg: github.com/linkedin/goavro/v2
BenchmarkNewCodecUsingV2-10                        98484             10709 ns/op
BenchmarkNativeFromAvroUsingV2-10                    513           2330489 ns/op
BenchmarkBinaryFromNativeUsingV2-10                 1555            747692 ns/op
BenchmarkNativeFromBinaryUsingV2-10                  582           2046254 ns/op
BenchmarkTextualFromNativeUsingV2-10                 262           4534945 ns/op
BenchmarkNativeFromTextualUsingV2-10                 240           4984041 ns/op
BenchmarkScanBinaryUsingV2-10                        618           1941487 ns/op
PASS
ok      github.com/linkedin/goavro/v2   10.183s

SpencerC avatar Jun 07 '23 16:06 SpencerC