C#: InvalidProtocolBufferException after adding new property

[kmiecikt]

trafficstars

What language / version do we use?

C#, .NET 6.0 running on Windows, with the following NuGets:

• Google.Protobuf 3.23.2
• Google.Protobuf.Tools 3.23.2
• Grpc.Tools 2.48.1

What do we do?

1. We added a new optional property to message exchanged by our gRPC microservices.
2. The microservices update existing property before sending it to other services (cannot reproduce without this step).
3. During the deployment, when we use both formats, we observed a spike of InvalidProtocolBufferException; some instances were also running out of memory.

Minimum standalone repro

Below is the minimum repro (V1 - the original message; V2 - includes new optional property):

1. Create V2 message and serialize it.
2. Deserialize it to V1 message.
3. Change a string property on the deserialized object and serialize it again.
4. Try to deserialize it to either V1 or V2 format. This step fails with InvalidProtoclBufferException.

Message definition V1:

syntax = "proto3";

message MessageContext {
	string name = 1;
}

Message definition V2:

syntax = "proto3";

message MessageContext {
	string name = 1;
	optional int64 newLong = 2; // New optional property
}

C# code that reproduces the problem

In the code below, the MessageContextOriginal is the V1 definition of the message (generated from Protobuf and renamed); the MessageContext is the class generated from V2 format.

using Google.Protobuf;

using var stream = new MemoryStream();

var v2 = new MessageContext
{
    Name = "original",
    NewLong = 123    // The newly added property that does not exist in V1 definition
};

v2.WriteTo(stream);

stream.Seek(0, SeekOrigin.Begin);
var v1 = MessageContextOriginal.Parser.ParseFrom(stream); // Deserialize it as V1 message
Console.WriteLine($"V2 -> V1: Passed {v1}");

v1.Name = "new"; // This is important - the we cannot reproduce exception without this step
stream.Seek(0, SeekOrigin.Begin);
v1.WriteTo(stream);

stream.Seek(0, SeekOrigin.Begin);
v2 = MessageContext.Parser.ParseFrom(stream); // Exception happens here: 'Protocol message contained a tag with an invalid wire type.'
Console.WriteLine($"V1 -> V2: Passed {v2}");

Exception

Google.Protobuf.InvalidProtocolBufferException
  HResult=0x80131620
  Message=Protocol message contained a tag with an invalid wire type.
  Source=Google.Protobuf
  StackTrace:
   at Google.Protobuf.UnknownFieldSet.MergeFieldFrom(ParseContext& ctx)
   at Google.Protobuf.UnknownFieldSet.MergeFieldFrom(UnknownFieldSet unknownFields, ParseContext& ctx)
   at MessageContext.pb::Google.Protobuf.IBufferMessage.InternalMergeFrom(ParseContext& input) in TestClasses.cs:line 262
   at Google.Protobuf.CodedInputStream.ReadRawMessage(IMessage message)
   at MessageContext.MergeFrom(CodedInputStream input) in TestClasses.cs:line 233
   at Google.Protobuf.MessageExtensions.MergeFrom(IMessage message, Stream input, Boolean discardUnknownFields, ExtensionRegistry registry)
   at Google.Protobuf.MessageParser`1.ParseFrom(Stream input)
   at Program.<Main>$(String[] args) in Program.cs:line 22

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[chaoyebugao]

Any upate?

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please reopen it.

This issue was closed and archived because there has been no new activity in the 14 days since the inactive label was added.