fast-sqlite3-inserts icon indicating copy to clipboard operation
fast-sqlite3-inserts copied to clipboard

`basic_async` not actually asynchronous ?

Open Zshoham opened this issue 4 years ago • 1 comments

I might be missing something here as I'm still new to async in rust, but as I understand it calling await on a future forces the runtime to serialize the execution of the program, this means that this code:

conn.execute("PRAGMA cache_size = 1000000;").await?;
conn.execute("PRAGMA locking_mode = EXCLUSIVE;").await?;
conn.execute("PRAGMA temp_store = MEMORY;").await?;

is completely synchronized and each call to the database needs to complete before the next call starts. Now, this seems fine as this only happens once outside of the benchmark loop.

But all the code in the faker function also uses await on each async call. For example this code :

let stmt_with_area = tx
    .prepare("INSERT INTO user VALUES (NULL, ?, ?, ?)")
    .await?;
let stmt = tx
    .prepare("INSERT INTO user VALUES (NULL, NULL, ?, ?)")
    .await?;

It seems to me that there is no reason the needs to be serialized and we should join these calls, in something like:

let (stmt_with_area, stmt ) = tokio::join!(
    tx.prepare("INSERT INTO user VALUES (NULL, ?, ?, ?)"),
    tx.prepare("INSERT INTO user VALUES (NULL, NULL, ?, ?)")
)

Finally, it also seems that we don't actually need to await the execution of the statements themselves inside the for but that would require saving all the futures in a big vector and then joining them, which seems a bit odd and would consume a lot of memory when running the for loop for 10e6 iterations.

Zshoham avatar Aug 30 '21 07:08 Zshoham

You can implement your idea of saving the futures into a vector but maybe not all of them. Maybe have a buffered 1000 items stream of futures to do that, I think performance might benefit quite a bit

blarfoon avatar Nov 13 '22 14:11 blarfoon