ion-python icon indicating copy to clipboard operation
ion-python copied to clipboard

C extension read/write lists of integers is slow

Open cheqianh opened this issue 2 years ago • 0 comments

Investigate why read/write lists of integers is slow. More specifically, read/write big integers and decimal is slow.

This is kind of duplicated to https://github.com/amzn/ion-python/issues/186 point three, but it's worth to have a separate issue to keep this performance issue on track.

E.g.

results = np.zeros((1000, 80))

# Ion
ion_list_dump = ion.dumps(results.astype("int32").tolist(), binary=True)
ion_list_load = np.array(ion.loads(ion_list_dump))

# JSON 
json_list_dump = json.dumps(results.astype("int32").tolist())
json_list_load = json.loads(json_list_dump)

cheqianh avatar Jun 09 '22 21:06 cheqianh