Zserio deserialization successful even with incomplete data

[Mallikarjuna-Mbrdi]

Zserio version and language Zserio: 2.1.4 Language: C++

Describe the bug Zserio deserialization successful even with incomplete data. in my application, i have data of n bytes to which i add a header info of m bytes and accidentally added just n bytes to the bitbuffer which is provided to zserioBitstreamReader, the deserialization still works.

cannot give further details, sorry. we can have a call any time. please let me know when we can discuss.

Thanks -Mallikarjuna

[mikir]

It depends where you add a header info.

If you add the header at the end, then this behavior is normal. The BitBuffer does not have to be parsed fully, there is no check if something is still left in the BitBuffer.

If you add the header at the beginning, then this behavior is strange and very unlikely to happen, but still possible. It can be that the binary data still can be parsed depending on the schema. For example, if schema does not have any constraints or optionals or enumerations or bitmasks, the probability of data misinterpretation is higher.

[Mallikarjuna-Mbrdi]

we are adding the header at beginning, is there a way to fix this issue, we need a predictable behavior in an ASIL context. is there a way to find inconsistency in the data?

[mikir]

We cannot see any way how to check fully the inconsistency of the binary data. I would say that it is up to application to detect full consistency.

But if you forgot to add Header in the binary data and the schema contains this Header, then the parsing should always fail (not enough data in the BitBuffer).

It would be a big help if you could share with us the minimal example which will reproduce your problem.

[Mallikarjuna-Mbrdi]

struct Message {
	bytes data;
};

struct Header{
        uint32 version;
	uint32 data_size;
	uint8 no_of_msgs;
        Message msg;
};

i have some schema similar to this, where i add header in the beginning and actual data following the header.

In my testing, lets say my msg size is 100bytes and the header size is 7 bytes.

Steps:

1. Serialize msg.
2. Add header to the message and serialize header.
3. Now, i tried to write the serialized data (header, which includes msg) to a std::array.
4. While writing to array, instead of writing header.size() bytes(in this case 107bytes), i have written msg.size() bytes(which is 100bytes).
5. Ended up missing last 7bytes of the msg.
6. Now use bitstreamreader to deserialize the data from std::array and was successful without an exception.
7. I found out this issue when i try to deserialize a smaller data of size 9 bytes + 7 bytes header, somehow with larger data in 10s of KB the deserialization worked fine.
8. And yes, the schema has enums, choice in it.

[mikir]

Something like this should work without any problem. I would double check if data used in the point "Now, i tried to write the serialized data to a std::array." are really Header not Message. If the data are really Header, then I would need more info to be able to investigate the problem further.

[Mallikarjuna-Mbrdi]

sorry, in point 3, data = header (which also includes the msg). Please let me know the info you require.

[mikir]

Would it be please possible to create minimal example (schema + minimal C++ application) which reproduces problem described by your points? Unfortunately, I am not able to guess from the current info where could be a problem.