pgliteworker with electron

[Venipa]

currently trying to create a worker so i can access the "proxy" via different workers. appearently the worker class mismatches with the source? because postMessage, addEventListener is not available

more main process logs:

(node:34988) UnhandledPromiseRejectionWarning: TypeError: a(...).addEventListener is not a function
    at (redacted)\node_modules\@electric-sql\pglite\dist\worker\index.cjs:8:3552
    at new Promise (<anonymous>)
    at new Ee ((redacted)\node_modules\@electric-sql\pglite\dist\worker\index.cjs:8:3526)
    at createDatabaseWorker ((redacted)\out\main\json-IHKDEz9o.js:916:14)
    at Object.<anonymous> ((redacted)\out\main\json-IHKDEz9o.js:923:16)
    at Module._compile (node:internal/modules/cjs/loader:1484:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1564:10)
    at Module.load (node:internal/modules/cjs/loader:1295:32)
    at Module._load (node:internal/modules/cjs/loader:1111:12)
    at c._load (node:electron/js2c/node_init:2:16955)
 

Source:

// my-pglite-worker.ts
import { PGlite } from '@electric-sql/pglite'
import { worker } from '@electric-sql/pglite/worker'
import { parentPort } from 'worker_threads'
const port = parentPort
if (!port) throw new Error('IllegalState')
worker({
  async init(options) {
    const meta = options.meta
    // Do something with additional metadata.
    // or even run your own code in the leader along side the PGlite
    return new PGlite({
      dataDir: options.dataDir
    })
  }
})

// createDatabaseWorker.ts
import { PGliteWorker } from '@electric-sql/pglite/worker'
import { isProduction } from '@shared/config'
import { app } from 'electron'
import path from 'path'
import { Worker } from 'worker_threads'
import PGWorkerPath from './worker?modulePath'
let client: PGliteWorker
const databasePath = isProduction
  ? path.join(app.getPath('userData'), 'data_app')
  : path.resolve(__dirname, '../database')
export default function createDatabaseWorker() {
  if (client) return client
  const worker = new Worker(PGWorkerPath, { type: 'module' })
  client = new PGliteWorker(worker as any, {
    dataDir: databasePath
  })
  return client
}

[Venipa]

after looking through the source, it seems pglite expects a "Web" Worker, so i tried to make a polyfill but due to pglite worker's source that requires navigator and tabChannel to exist this wont do :/

i guess this turns into a request to support node's "worker_threads"?

[samwillis]

Hey @Venipa

Yes, we should 100% create a PGliteWorker for use with Node and electron. I suspect it would want to be a separate import, maybe sharing some RPC code. It's on the list!

[Deliana90]

looking forward ^_^

[joaocc]

Hi. Any news on this issue? Thanks

[Sandakan]

Is there any update on this? I noticed some UI hangs due to complex table relationships and large data, and this improvement would help resolve those issues.

[maxwhipw]

Bump, encouragement for prioritization is I have been looking everywhere for a cross-platform embeddable database with compatible sync libraries and there seem to be a lot of people out there who are interested as well.

I think it's one of the most useful use cases of this tech, that is client-side cross platform compatible db storage without size limits like in the case of IndexedDB. I'm surprised Electron doesn't have existing options for this yet but this project seems like the best hope at present