pycapnp icon indicating copy to clipboard operation
pycapnp copied to clipboard

Pickling of `_DynamicListReader` objects is silently broken

Open jcpetruzza opened this issue 8 years ago • 0 comments

While it is possible to pickle an instance of _DynamicStructReader (which uses the capnp bytes under the hood), if one tries to pickle instead any of the list attributes of the object, the pickling succeeds, but the deserialized list will be empty.

The problem seems to be that there is no suitable __reduce_ex__() method defined for _DynamicListReader and the default one doesn't work well.

To reproduce, assume we have a schema s with the following definition:

struct Foo{
    stuff @0: List(UInt64);
}

Then we can do:

>>> foo = s.Foo.new_message()
>>> foo.stuff = [1, 2, 3]
>>> foo = s.Foo.from_bytes(foo.to_bytes())
>>> foo.stuff
<capnp list reader [1, 2, 3]>
>>> import pickle
>>> roundtrip = lambda o: pickle.loads(pickle.dumps(o, pickle.HIGHEST_PROTOCOL))
>>> assert list(foo.stuff) == list(roundtrip(foo).stuff)
>>> assert list(foo.stuff) == list(roundtrip(foo.stuff))
AssertionError
>>> roundtrip(foo.stuff)
<capnp list reader []>

jcpetruzza avatar Oct 18 '17 14:10 jcpetruzza