pycardano
pycardano copied to clipboard
Support decoding list/indefinite list from cbor
tl;dr: This PR adds support for proper parsing of fixed vs indefinite lists from CBOR.
Background Currently, pycardano uses a custom encoder to embed indefinite lists when encoding classes to cbor, even if the length of the list is less than 31 elements.
This creates an issue when decoding with cbor2
, because cbor2
decodes all lists to the list
class without any indication of whether the list was encoded as a fixed/indefinite list. This loss of information is problematic because hashes are frequently generated off of CBOR, so encoding as a fixed list vs indefinite list generates a different hash even if the content of the datum is the same.
Implementation Detail
This PR is unfortunately dependent on a minor fix in cbor2
. The CBORDecoder
class doesn't have a custom decoding mechanism to make modifications to how data is decoded. One approach to fixing this issue would be to subclass CBORDecoder
to intervene when an indefinite list is found. However, the CBORDecoder
was implemented in such a way that it relies on a global dictionary that references internal method definitions on a class, making subclassing challenging. To resolve this issue a PR was opened to cbor2
that simply moves the global dictionaries inside the CBORDecoder
class, which has no impact on any other section of code:
https://github.com/agronholm/cbor2/pull/225
This PR takes advantage of that PR to subclass the CBORDecoder
and intervene in the case of an indefinite list. When an indefinite list is discovered, it is cast to IndefiniteList
and returned, while all other lists are returned as a standard list
. This required some additional changes, specifically a change how data is being decoded on the pycardano
side.
This should fix a few different Issues, and fixed a previously undiscovered issue related to roundtrip cbor encoding of datums (datum -> cbor ->datum -> cbor).
fixes #311 fixes #330