json2csv icon indicating copy to clipboard operation
json2csv copied to clipboard

Use all available cores

Open simonschmidt opened this issue 10 years ago • 0 comments

json2csv uses a lot of CPU, but only one core. json.Unmarshal seems to be taking up most of the time so I made a forked version that launches a whole bunch of them in parallel

A 207Mb json file with 241804 lines takes 15s to convert to csv with the single-threaded version, using all cores (2 real + 2 hyperthreading) it takes 7s. However using just one core on the multi-threaded version takes 17s It might be worth to keep the single-threaded version in the code and use that when running on one core.

parallel version

simonschmidt avatar Jan 08 '14 14:01 simonschmidt