Dexie DB within a web worker

[ballabusta]

First of all, thank you for this IDB wrapper, the API is rich and well documented.

I am having some issues with opening (with the purpose of writing) an existing dexie database from a web worker (same origin). The goal is to fetch a significant amount of data (500k+) and then write in chunks to indexedDB. The fetch part works well in the worker, however, opening the existing database fails. I have also tried to create a new database from the worker, and then initiated a db.tableName.bulkPut but that new database would not even be created (not visible in Chrome > devTools > Applications tab)

Here's what I am currently doing:

• In window context:

import Dexie from "dexie"; const db = new Dexie("myDB"); db.version(1).stores({ table1: "++key, AColumnName1", table2: "++key, AColumnName2", table3: "AColumnName3" }); export default db;

• in worker context:

const initialLength = scenarioPostProcess.length; // Getting the data in scenarioPostProcess Dexie.getDatabaseNames().then((names) => { log(Here are the names ${names}); });

new Dexie("myDB").open().then((db) => { while (scenarioPostProcess.length) { const bulkPutPromise = db.table1.bulkPut(scenarioPostProcess.splice(0, Math.floor(initialLength * 0.15))); bulkPutPromise.then((lastKeyInsertInDb) => { if (lastKeyInsertInDb) { log(The last key inserted in table is ${lastKeyInsertInDb}); } }); }
}).catch((error) => { log(Issue with database: ${error}); }); Using React with JS ES6, webpack with worker-loader Dexie 2.0.4

Any ideas would be greatly appreciated.

[opavader]

Facing the same issue myself. working fine from main thread but not in worker. I am using the worker-loader with webpack as well.

[dfahlander]

@ballabusta In the worker schenario, your example code does not specify version().stores(). That is possible, but then you need to access the tables through db.table(tableName) rather than just db.tableName as Dexie only populates the implicit table properties when in version().stores().

[dfahlander]

It's often better to declare the db instance in its own module and import the same module from both worker and web.

// db.js
import Dexie from "dexie";

export const db = new Dexie("myDB");
db.version(1).stores({
  table1: "++key, AColumnName1",
  table2: "++key, AColumnName2",
  table3: "AColumnName3"
});

// worker.js
import { db } from './db'; // You get a db with property table1 attached (because the schema is declared)

self.onmessage = event => {
  ...
  db.table1.bulkPut(...).then(lastId => {
    ...
  }).catch(error => {
    ...
  });
}

[opavader]

Thnx @dfahlander, the db.table('someTable') approach seems to work. Not sure what I am doing wrong here that table fields are not getting populated.

const LATEST_VERSION = 1;

class ArchDB extends Dexie {
    eois: Dexie.Table<LDefs.IEntryEOI, string>; 
    alerts: Dexie.Table<LDefs.IEntryAlerts, string>;

    constructor(dbSuffix: string) {
        super('ArchDB_' + dbSuffix);
        // using unique id strings to prevent duplicates
        this.version(LATEST_VERSION).stores({
            eois: '&id, updated, StartTime, EndTime', // Only want index by StartTime & EndTime as rest of the filtering is done in crossfilter
            alerts: '&id, updated, opentime, closetime, acktime',
        });
    }
}

export interface IDriver {
    db: ArchDB;
    ready: boolean;
}

const driver: IDriver = {
    db: null,
    ready: false,
};


export const initDriver = (domain: string) => {
    console.log('initializing local data store for : ' + domain);

    driver.db = new ArchDB(domain);

    driver.db.open().catch(function (err) {
        console.error('Failed to open db: ' + (err.stack || err));
    }).then(() => {
        driver.ready = true;
    });
};

I am checking ready to be true before accessing the table fields.

[slominskir]

It appears Firefox doesn't support import of modules in web workers, so the example above doesn't work in Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1247687

[slominskir]

Also worth mentioning that a web worker appears to have a separate module map vs the main script. The ES6 module will be executed twice if for example your web worker imports the db.js for writing to the IndexDB and your GUI running on the main script context imports db.js to read from IndexDB. This can cause a blocked situation if you're attempting to manually drop the database on version upgrades.

[dfahlander]

Closing this issue. Dexie is supported in both web workers, shared workers and service workers (since many years ago). This issue was about how to import it, which is more of an issue with web workers itself and the module system of plain web vs bundlers.

To anyone using dexie in a web worker, I recommend to declare the db instance in its own module and import the same module from both worker and web. They will of course be different instances though (workers run in different threads and do not share instances). The point is that code can be declared in a single place and they both work and observe the same data!

For this to work with a module setup, I still recommend using bundler such as webpack, vite, rollupjs, esbuild etc even though it could also be achieved using vanilla javascript but if you want db declared in a module, you'll need to mark the worker with {type: "module}when creating it from your web code and you need to use native es modules from your web javascript as well and import it as so `import Dexie from './path-to/dexie.mjs'.

Vanilla setup without modules is also possible (but feels a bit old fashioned and doesn't scale so well for large apps).