purescript-protobuf icon indicating copy to clipboard operation
purescript-protobuf copied to clipboard

ArrayView instead of Array for repeated scalar fields

Open jamesdbrock opened this issue 4 years ago • 2 comments
trafficstars

I wonder if it would be a worthwhile performance improvement to represent repeated scalar fields with ArrayView instead of Array.

For example a repeated double message field would be type Float64Array instead of Array Number.

We already have special cases for packed repeated scalar fields so maybe this would be easy.

jamesdbrock avatar Jun 24 '21 00:06 jamesdbrock

When deserializing a packed repeated scalar field we don't know how many repeated elements are in the field in advance, so we wouldn't be able to preallocate exactly the right size ArrayView. We would have to make a growable ArrayView buffer like https://pursuit.purescript.org/packages/purescript-dynamic-buffers

jamesdbrock avatar Jun 24 '21 00:06 jamesdbrock

When deserializing a packed repeated scalar field we don't know how many repeated elements are in the field in advance,

Wait that's not quite true, we know the length of the array, and if the element type is fixed-width (double, float, fixed32, fixed64, sfixed32, sfixed64) then we can just divide by the byte width of the array element type to get the number of repeated elements.

That still leaves the problem of variable-width-type packed repeated fields though.

jamesdbrock avatar Jul 28 '21 14:07 jamesdbrock