timescaledb-parallel-copy icon indicating copy to clipboard operation
timescaledb-parallel-copy copied to clipboard

Published 20 hours ago • verified publisher icon timescale

Performance tuning tips

Open brettwooldridge opened this issue 3 years ago • 1 comments

This is rather an open question. I am looking for any experience/tips anyone has regarding performance tuning to increase import speed. We are trying to import several billion rows of data from InfluxDB to TimescaleDB.

Batch size? Copy options? PostgreSQL tuning parameters?

brettwooldridge avatar Feb 04 '22 05:02 brettwooldridge

Here are a few good tips: https://www.timescale.com/blog/13-tips-to-improve-postgresql-insert-performance/

Also, there's some good ideas for large datasets here: https://github.com/timescale/timescaledb-extras#useful-utilities

I'd also encourage you to enable the --report and start testing with different batch sizes and parallel processes. It will depend on the machine you're using.

jonatas avatar Mar 07 '22 12:03 jonatas