pypyodbc icon indicating copy to clipboard operation
pypyodbc copied to clipboard

Performance issue

Open dhanababum opened this issue 7 years ago • 2 comments

Hi, I am very glad for pypyodbc and using pypyodbc for my day to day work. I work with 5-10GB data, will apply computations and then insert in MS Access using pypyodbc.

Observed some performance variations following code,

cur.execute("Insert")
con.commit()

Inserting row by row using above code and it is very slow. Then I came up with

cur.executemany("Bulk insert")
con.commit()

But it is little fast compared with row by row insertion, But I need extra performance :). Could you give me any suggestions for me

dhanababum avatar Apr 29 '17 17:04 dhanababum

As you have apparently discovered, INSERT INTO ... SELECT can offer significant performance advantages over row-by-row INSERTs, even if those individual INSERTs are "batched" (e.g., by executemany in pypyodbc or by executeBatch in JDBC).

gordthompson avatar Jul 30 '17 15:07 gordthompson

@gordthompson Sorry for late reply. Yes I found.

dhanababum avatar Sep 15 '17 18:09 dhanababum