protobuf
protobuf copied to clipboard
protocolbuffers
C# .NET deserialization of message that contains repeated float field, decodes incorrectly sometimes (based on actual data sent)
What version of protobuf and what language are you using? Version: gRPC version 1.47.1 (which includes the grpc_cpp_plugin.exe & grpc_csharp_plugin.exe files) Language: C++ (Server + client) & C# (client)
What operating system (Linux, Windows, ...) and version? Windows 10 x64 v21H2
What runtime / compiler are you using (e.g., python version or gcc version) Visual Studio Professional 2019 v16.11.11
What did you do? Create proto with message containing repeat float field.
For example, see State_Data below inside .proto file:
myProto.proto
syntax = "proto3";
package test;
service DataServiceAPI {
rpc NotifyStateChange(Scene_DataState) returns (Empty) {}
}
message Empty {
}
message State_Data
{
repeated float clear_color = 1;
uint64 modifiedFlag = 2;
}
if I send sateData with 4 floats, it seems that the floats received on the other end (C# .NET client), depends on the content sent & is non deterministic.
What did you expect to see Info de-serialized properly as sent on C++ side, no matter what value sent.
What did you see instead? value depends on the value sent, sometimes decodes incorrectly...
for some values, the deserialization is correct, example: if I send 0 -> I get 0, if I send 0.5 -> I get 0.5
however for other values, the deserialization is incorrect, example: if I send 1.0 -> I get 0.74
Note: that when using C++ client for reference, there it decode data properly every time ! so this means the server send that data serailized data correctly !
Anything else we should know about your project / environment
The C# .NET is using Nuget version v3.21.4
Note: that when using C++ client for reference, there it decode data properly every time ! so this means the server send that data serailized data correctly !
Not necessarily - it's entirely possible for the C++ code to have "equal and opposite" bugs in serialization. (Likewise if this turns out to be C# bug, I suspect that C# to C# would always serialize/deserialize correctly...)
That said, it's more likely that this is a C# bug than a C++ bug simply due to the usage that C++ protobufs get within Google.
In order to diagnose this further, it would be really useful to remove gRPC from the equation. If you could:
- Create a
State_Datamessage in C++ with a value of 1.0 - Save the binary wire format representation to a file
- Load the file from C# and display the incorrectly-parsed value
... then we should be able to reproduce the problem easily - because you'll be able to supply the data file that causes the problem. We can look at the exact bytes to work out what they represent according to the spec, rather than according to any given implementation.
@obi0ne: Any updates on this? (Without any more information, I'll probably close this issue some time next week.)
Hi, sorry takes long to respond. I will provide with requested data to debug this one in a day or two. Last week was a over busy week at work I didn't had time to reply. I hope to post all the required data for debug this Sunday.
sorry again for late response
i'm working on reproducable clean code (using conan & cmake & nuget) code that will demo the problem, will have it ready by Sunday afternoon. (weekend here, will continue work on it on Sunday) Code will be available on github public repo @ https://github.com/obi0ne/protobuf_bug_reproduce Once I'll have code finished, bug reproduced, i'll reply back here with more info as requested.
That's great, thanks. (I won't be working on Sunday or Monday, but I'll look on Tuesday.) If you're able to include sample output data files so that I don't need to build the C++ unless I really have to, that would be appreciated :)
just finished standalone coding (code is visible on repo link above), however the bug I saw does not reproduce :(
On both directions C++ -> C# .NET and C# .NET -> C++
sending 1.0f decodes fine as 1.0f on other end, on both sides...
does it means it's a gRPC bug ?
It's possible - hard for me to say, to be honest.
Are you able to reproduce with a C# gRPC server and client? If it has to be a C++ server and a C# client, that's somewhat harder for me to reproduce, but obviously I'll still do so if that's the only way of reproducing it. It's just easier if it's C# on both sides :)
Side-note: thanks ever so much for engaging and helping with the diagnosis. It's much appreciated, especially when it doesn't then repro easily :(
Not sure, i will try to enter gRPC into equation and i will update the results.
Will first try both on c#, and if not work will try mix with c++.
Tomorrow i’m not sure i will have time for performing this test, please allow me time till Tuesday to add gRPC code and will update if i can provide with some reproducable code that shows the problem.
Tomorrow i’m not sure i will have time for performing this test, please allow me time till Tuesday to add gRPC code and will update if i can provide with some reproducable code that shows the problem.
Of course. I should mention that I'm away on Wednesday and Thursday as well as Monday - but I'm sure we'll get there in the end.
having some stress period at work, if possible please keep this issue open, the repro with gRPC will be made over weekend, and not by end of today, as I initially believed in.
Sorry for the delay
Hi, i encounter a problem that performs like this (direction is C++ -> C#): for some values, the deserialization is also incorrect, example: if I send data_size:10 -> I get dataSize:10, the key field will be changed. I wonder the essential reason why doing so as well. Looking forward to the reply, thank you!
@greyDoll: That sounds like it's different and may well be just related to the JSON format. Please file a new issue with a lot more detail, specifically what you mean by "I send" and "I get".
@obi0ne: Any updates on this? Don't worry if you haven't had time to work on it at all - just wondering.
sorry I have no updates yet, we had holidays here. I will try to code something on this Monday/Tuesday.
Thanks again for your patience and understanding - i'm sorry for the delay in response.
Closing as obsolete - if you run up against this in a reproducible way, we can reopen.