cmlkit
cmlkit copied to clipboard
Drop yaml dumper for son, replace with json
JSON is comically faster. Here is a benchmark for dumping/loading 2000 small dicts (3 repeats):
JSON:
{'times': array([0.38685818, 0.34857985, 0.34886317]), 'mean': 0.36143373133333334, 'min': 0.3485798470000001, 'max': 0.3868581750000001}
Yaml:
{'times': array([ 9.26613551, 10.8498244 , 9.88484843]), 'mean': 10.000269449333334, 'min': 9.266135509, 'max': 10.849824405}
This really hurts when doing Run.checkout()
.
For dumping numpy stuff with json, use the hilde
stuff, https://gitlab.com/flokno/hilde/blob/cli/hilde/helpers/converters.py.