gRPC Reflection Call Fails to Recognize optional Keyword in Proto Definitions

[guzhijun369]

Problem Description: When using Python for gRPC reflection calls, fields defined with the optional keyword in the proto file are being filtered out when default values are passed. This causes the server to not receive the fields, leading to incorrect logic handling. Details:

Proto Definition: The relevant field is defined with optional

Client-Side Implementation: In the Python client code, when passing parameters and processing

after calling ParseDict for serialization, the status field (marked as optional in proto) is treated as a default value and filtered out. Unexpected Behavior: Manual attempts to add the status field fail. Server-side debugging shows __pbn__status is null, indicating the field is considered "not passed"

This issue prevents the server from receiving optional fields with default values during reflection-based calls, breaking intended functionality. Any insights or fixes for preserving optional fields during serialization/transmission would be greatly appreciated.

Protobuf version: python protobuf/v6.32.0 Client Language:Python3.9 Linux, Windows

[sergiitk]

cc gRPC Python @sreenithi @asheshvidyut

[jguamie]

@guzhijun369 We are attempting to reproduce this. Can you please provide the following (this is from our bug report template)? Also, some repro code would be helpful as well.

What did you do? Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

[guzhijun369]

client.py The attached document contains my source code, and this is how I made the call:

When the value of status is 0, the backend fails to recognize it. However, values 1, 2, and 3 work normally. This issue occurs because 0 is the default value and thus gets filtered out. The following is the Protobuf definition of status:

Additionally, when I use direct Protobuf calls instead of reflection calls, the backend can recognize status with the value 0 correctly.

[sreenithi]

Thanks for the client repro code. It was still partially incomplete without the full proto and server to run the code, and hence I created a mock server and added some more details in the proto files here: https://github.com/sreenithi/repro_protocolbuffers_23299

I also tried adding some DEBUGGING statements at different places like ReflectionClient.get_file_descriptors_by_symbol, BasicGrpcClient._request and MessageParsers.parse_request_data to check where status may be getting lost.

However with the mock server I'm using, I see that the status value is always present and hence not able to reproduce the issue. Can you share your server code as well or try with similar DEBUGGING lines in your codebase and share the output, to help investigate this further.

[guzhijun369]

I can indeed make calls using your server-side, and the parameters received by the server include status. However, our company's backend framework is C++, and there are differences in the reflection source data obtained during calls, with the has_presence attribute being false. I need to wait for the backend developer to write a demo for me and then provide their code, so please allow some time for this.