Akumuli
Akumuli copied to clipboard
parallel load data
Hi, I use influxdb-comparisons which tool you developed. In load_data.sh file ,I set workers>1, where is "broken pipe" error running. I find "late write" in log. Because I loading data's timestamp is smaller than database data? If use parallel loading ,this error will still exist. Do you have good idea for parallel loading.
This can happen if you haven’t cleared the database between the runs of the ingestion tool from the influxdb-comparisons. It'll try to write the same data twice. And, I believe that the 'influxdb-comparisons' doesn't work in parallel. I have this set of scripts that can be used to write data in parallel, if you want to benchmark the top ingestion performance - https://github.com/Lazin/roundtrip_test. I think it's easier to use than the tool you mentioned.
BTW, I'm planning to add Akumuli support to the https://github.com/timescale/tsbs test suite. It will be easier to use and it will work in parallel.
Thanks, I'll have a try.