to_dict is returning some missing values

[lindycoder]

Hello!

It would seem that there is an behaviour inconsistency based on the type of a struct member when deserializing.

foo.Foo.from_bytes(serialized)

Expected behaviour

Deserializing a message that was serialized missing values should miss the same values.

Actual behaviour

Text values are correctly missing and Int values are set to 0

Notes

Here's a little test I wrote (fits in test_serialization.py)

@pytest.mark.parametrize('content', [
    # Real values, works fine
    {'id': 123, 'name': 'wat'},
    # Values that match what verbose=True returns when missing, works fine
    {'id': 0, 'name': ''},
    # No values at all : will only return the id, which is an integer
    {},
])
def test_deserializing_missing_attributes(content):
    foo = capnp.load(os.path.join(this_dir, 'foo.capnp'))

    serialized = foo.Foo.new_message(**content).to_bytes()

    deserialized = foo.Foo.from_bytes(serialized)

    assert deserialized.to_dict() == content

The 3rd test fails

>       assert deserialized.to_dict() == content
E       AssertionError: assert {'id': 0} == {}
E         Left contains 1 more item:
E         {'id': 0}
E         Use -v to get the full diff

What do you think?

[haata]

The serialization seems to be different.

{'id': 0, 'name': ''}
b'\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x01\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

{'id': 0}
b'\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x01\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

This indicates that there are 8 extra bytes (minimum segment size) in the first example.

I have two guesses

1. Serialization is wrong (that's where id=0 starts)
2. Text fields are handled differently when unset (I haven't been able to locate any code for this)