Make select Faster

[mmontagna]

What

Use Psycopg2 directly for PG selects in order to improve performance.

Why

sqlalchemy ads ~1ms of additional latency for basic selects and more for queries which return large result sets.

Without sqlalchemy

➜  test git:(marco-make-select-faster) ✗ lore python sql_test.py
...
raw rows average runtime 0.23196000000000003 ms

With sqlalchemy

➜  test git:(marco-make-select-faster) ✗ lore python sql_test.py
...
raw rows average runtime 1.371959 ms

sql_test.py

import lore, lore.io
import time
import numpy as np
runtimes = []

main = lore.io.main


for item_id in range(0, 10000):
    st = time.time()
    results = main.select(sql='select 1 where 1 = 1;')
    runtime = round((time.time() - st)*1000.0, 2)
    runtimes.append(runtime)

avg_runtime = np.mean(runtimes)

print("raw rows average runtime {} ms".format(avg_runtime))