deepkit-framework icon indicating copy to clipboard operation
deepkit-framework copied to clipboard

Concurrent SQLite :memory: queries cause crashes

Open timvandam opened this issue 3 years ago • 1 comments

Test suite:

import { Database } from '@deepkit/orm';
import { SQLiteDatabaseAdapter } from '@deepkit/sqlite';
import { AutoIncrement, PrimaryKey } from '@deepkit/type';

class User {
  id: number & PrimaryKey & AutoIncrement = 0;
  constructor(public username: string) {}
}

// does not pass
it('works concurrently in :memory:', async () => {
  const db = new Database(new SQLiteDatabaseAdapter(':memory:'), [User]);

  await db.migrate();

  await expect(
    Promise.all(Array.from({ length: 1000 }, () => db.persist(new User('test')))),
  ).resolves.not.toThrow();
});

// passes!
it('works sequentially in :memory:', async () => {
  const db = new Database(new SQLiteDatabaseAdapter(':memory:'), [User]);

  await db.migrate();

  for (let i = 0; i < 1000; i++) {
    await expect(db.persist(new User('test'))).resolves.not.toThrow();
  }
});

// passes!
it('works concurrently in file', async () => {
  const db = new Database(new SQLiteDatabaseAdapter('/tmp/a.sqlite'), [User]);

  await db.migrate();

  await expect(
    Promise.all(Array.from({ length: 1000 }, () => db.persist(new User('test')))),
  ).resolves.not.toThrow();
});

// passes!
it('works sequentially in file', async () => {
  const db = new Database(new SQLiteDatabaseAdapter('/tmp/b.sqlite'), [User]);

  await db.migrate();

  for (let i = 0; i < 1000; i++) {
    await expect(db.persist(new User('test'))).resolves.not.toThrow();
  }
});

timvandam avatar Sep 05 '22 14:09 timvandam

I believe this is caused by SQLiteConnectionPool: it creates multiple SQLiteConnection instances which all have their own connection to :memory:. This means that each instance will be connected to its own in-memory database. Only the first of these is migrated, hence the first test case throws a no such table error

To fix this the connection pool should be disabled (i.e. adapter.connectionPool.maxConnections=1) when using :memory:.

Setting adapter.connectionPool.maxConnections = 1 also does not work currently due to a off-by-one error: https://github.com/deepkit/deepkit-framework/blob/master/packages/sqlite/src/sqlite-adapter.ts#L206. Setting it to 0 works though.

Will make a PR for this all.

timvandam avatar Sep 06 '22 17:09 timvandam