ts-proto icon indicating copy to clipboard operation
ts-proto copied to clipboard

Optional repeated enum field encoding/decoding issue

Open jeffnsquared opened this issue 2 years ago • 1 comments

I'm running into an issue decoding a Protobuf message and I suspect its because of how ts-proto is encoding the message. I have a Proto below

enum OfferType {
  OFFER_TYPE_UNSPECIFIED = 0;
  OFFER_TYPE_COUPON = 1;
  OFFER_TYPE_DISCOUNT = 2;
}

message OfferFilter {
  repeated string tokens = 1;
  repeated OfferType types = 2;
}

and in Typescript, I'm building it with

offerFilter: {
  tokens: [],
  types: [],
}

The problem is my decoding logic (based on https://github.com/square/wire) attempts to decode the types field expecting a value. This issue doesn't exist for the tokens field. It only occurs for optional repeated enum fields.

The decode error is below

java.io.IOException: Expected to end at 61 but was 62
at ProtoReader.afterPackableScalar(ProtoReader.kt:385)
at ProtoReader.readVarint32(ProtoReader.kt:277)
at EnumAdapter.decode(EnumAdapter.kt:72)
at EnumAdapter.decode(EnumAdapter.kt:25)
at protos.api.Filter$Companion$ADAPTER$1.decode(OfferFilter.kt:218)

So if I actually pass in the value like so, the decoding works -

offerFilter: {
  tokens: [],
  types: [OfferType.OFFER_TYPE_COUPON],
}

I'm on Proto2 for what its worth. My questions are

  • Is this expected behavior from ts-proto perspective?
  • I'm trying to log the binary / encoded message - is there a way to do this?

I attempted to do

EntityFilter.encode({
    offerFilter: {
	tokens: [],
	types: [OfferType.OFFER_TYPE_COUPON],
    }
});

but printing it is giving me values such as {\\n len: 11,\\n head: Op {\\n fn: [Function: noop] which I'd assume is wrong. I'm trying to get the encoded message and compare across caller and receiver services.

  • Am I missing an opt when running the command?

Thank you!

jeffnsquared avatar Apr 21 '23 18:04 jeffnsquared

Hi @jeffnsquared ; I'm wondering if maybe wire does not expect the field to be serialized at all when empty.

You can see an example of how ts-proto encodes a list of enums here:

https://github.com/stephenh/ts-proto/blob/main/integration/simple/simple.ts#L351

    writer.uint32(66).fork();
    for (const v of message.oldStates) {
      writer.int32(v);
    }
    writer.ldelim();

You could potentially just hand-edit the code to only do the fork + ldelim if the message.oldStates or offerFilter.types is non-empty.

The code that generates this section of encode is here:

https://github.com/stephenh/ts-proto/blob/main/src/main.ts#L1367

Kinda odd, b/c no one has complained about this before, so would be interesting if you could dig into whether this is a Wire-specific convention or something in the protobuf spec that we're missing.

{\\n len: 11,\\n head: Op {\\n fn: [Function: noop]

My guess is that is just what Buffer console.logs out as, and you probably what to toByteArray it or something (I forget the methods off hand).

stephenh avatar Apr 25 '23 00:04 stephenh