python-betterproto icon indicating copy to clipboard operation
python-betterproto copied to clipboard

Zeroes are not serialized with bytes() in proto2 syntax with required field

Open VityaSchel opened this issue 5 months ago • 3 comments

Summary

There should be an alternative bytes method that allows to serialize zeroes for proto2 syntax. Currently zeroes enums are skipped as they are "default" values which is incorrect because unless the field is optional, it must be present in encoded form.

Reproduction Steps

  1. Create a simple proto2 file:
message MyMessage {
    enum MyEnum {
        FOOBAR = 0;
        BAZ = 1;
    }
}
  1. Compile to python:
@dataclass
class MyMessage(betterproto.Message):
	# @required
	action: "MyEnum" = betterproto.enum_field(1)

class MyEnum(betterproto.Enum):
	FOOBAR = 0
	BAZ = 1
  1. Use the FOOBAR value (0):
print(
	MyMessage(
		action=MyEnum.FOOBAR,
	)
)

You can see the MyEnum.FOOBAR here. However when you try to serialize it, the encoder skips this value incorrectly thinking it's default and hence shouldn't be added:

print(
	MyMessage(
		action=MyEnum.FOOBAR,
	).to_json()
)
# or
print(
	MyMessage(
		action=MyEnum.FOOBAR,
	).to_dict()
)
# or
print(
	bytes(MyMessage(
		action=MyEnum.FOOBAR,
	))
)

This can only be "fixed" by passing include_default_values=True in to_json or to_dict:

print(
	MyMessage(
		action=MyEnum.FOOBAR,
	).to_json(include_default_values=True)
)

But I need binary representation of the MyMessage class

Expected Results

N/A

Actual Results

N/A

System Information

Latest beta version

Checklist

  • [x] I have searched the issues for duplicates.
  • [x] I have shown the entire traceback, if possible.
  • [x] I have verified this issue occurs on the latest prelease of betterproto which can be installed using pip install -U --pre betterproto, if possible.

VityaSchel avatar Jul 25 '25 21:07 VityaSchel

@AdrienVannson @Gobot1234 I see you guys working on a new version and the same bug is present in your betterproto2 fork as well:

https://github.com/betterproto/python-betterproto2/blob/59f7441de598c1348d30becd289a6bb112417a40/betterproto2/src/betterproto2/init.py#L762-L764

I would only consider this to be non bug if Google's protobuf.js implementation allowed to decode these required values as zeroes if they're not present. Unfortunately, protobuf.js decode method throws "missing required 'action'" error if I attempt to decode binary representation encoded by betterproto.

VityaSchel avatar Jul 25 '25 21:07 VityaSchel

Yes, only proto3 is supported for now

AdrienVannson avatar Jul 26 '25 07:07 AdrienVannson

I eventually used the meta.default_gen workaround if anyone stumbles upon it in the future

VityaSchel avatar Jul 26 '25 15:07 VityaSchel