cmlkit icon indicating copy to clipboard operation
cmlkit copied to clipboard

Drop yaml dumper for son, replace with json

Open sirmarcel opened this issue 5 years ago • 1 comments

JSON is comically faster. Here is a benchmark for dumping/loading 2000 small dicts (3 repeats):

JSON:
{'times': array([0.38685818, 0.34857985, 0.34886317]), 'mean': 0.36143373133333334, 'min': 0.3485798470000001, 'max': 0.3868581750000001}
Yaml:
{'times': array([ 9.26613551, 10.8498244 ,  9.88484843]), 'mean': 10.000269449333334, 'min': 9.266135509, 'max': 10.849824405}

This really hurts when doing Run.checkout().

sirmarcel avatar Jun 12 '19 11:06 sirmarcel

For dumping numpy stuff with json, use the hilde stuff, https://gitlab.com/flokno/hilde/blob/cli/hilde/helpers/converters.py.

sirmarcel avatar Jun 12 '19 11:06 sirmarcel