pycardano icon indicating copy to clipboard operation
pycardano copied to clipboard

Support decoding list/indefinite list from cbor

Open theeldermillenial opened this issue 11 months ago • 6 comments

tl;dr: This PR adds support for proper parsing of fixed vs indefinite lists from CBOR.

Background Currently, pycardano uses a custom encoder to embed indefinite lists when encoding classes to cbor, even if the length of the list is less than 31 elements.

This creates an issue when decoding with cbor2, because cbor2 decodes all lists to the list class without any indication of whether the list was encoded as a fixed/indefinite list. This loss of information is problematic because hashes are frequently generated off of CBOR, so encoding as a fixed list vs indefinite list generates a different hash even if the content of the datum is the same.

Implementation Detail This PR is unfortunately dependent on a minor fix in cbor2. The CBORDecoder class doesn't have a custom decoding mechanism to make modifications to how data is decoded. One approach to fixing this issue would be to subclass CBORDecoder to intervene when an indefinite list is found. However, the CBORDecoder was implemented in such a way that it relies on a global dictionary that references internal method definitions on a class, making subclassing challenging. To resolve this issue a PR was opened to cbor2 that simply moves the global dictionaries inside the CBORDecoder class, which has no impact on any other section of code: https://github.com/agronholm/cbor2/pull/225

This PR takes advantage of that PR to subclass the CBORDecoderand intervene in the case of an indefinite list. When an indefinite list is discovered, it is cast to IndefiniteList and returned, while all other lists are returned as a standard list. This required some additional changes, specifically a change how data is being decoded on the pycardano side.

This should fix a few different Issues, and fixed a previously undiscovered issue related to roundtrip cbor encoding of datums (datum -> cbor ->datum -> cbor).

fixes #311 fixes #330

theeldermillenial avatar Mar 14 '24 02:03 theeldermillenial