icews icon indicating copy to clipboard operation
icews copied to clipboard

Faster inserts on update

Open andybega opened this issue 4 years ago • 0 comments

Updating the DB takes a lot of time right now. SQLite doesn't have bulk inserts, but it seems that using a transaction for bulk inserts might be faster?

  • Use transactions for bulk inserts: https://stackoverflow.com/Questions/364017/faster-bulk-inserts-in-sqlite3
  • Transactions in RSQLite: https://rsqlite.r-dbi.org/reference/sqlite-transaction.html

So maybe wrapping all the raw file inserts in a transaction would increase the update speed.

andybega avatar Oct 25 '21 10:10 andybega