npm compat: unable to use "npm:drizzle-kit" cli tool

[andrewbrey]

Note Demo / reproduction repository: https://github.com/andrewbrey/deno_npm_compat_drizzle-orm

---

When attempting to use the drizzle-orm (https://orm.drizzle.team) tool called drizzle-kit to generate sql schema migrations from TypeScript model definitions via npm-specifiers in Deno, I encountered the following error:

error: Uncaught TypeError: worker.unref is not a function
    at <anonymous> (file:///home/andrew/.cache/deno/npm/registry.npmjs.org/esbuild/0.18.12/lib/main.js:2295:10)

The linked repository ( https://github.com/andrewbrey/deno_npm_compat_drizzle-orm ) contains a minimal setup needed to test out if drizzle-orm works with Deno npm-specifier based compatibility. The main point of interest is actually the migrate task defined in the deno.jsonc:

// deno.jsonc

{
  "tasks": {
    ...
    "migrate": "deno run -A npm:drizzle-kit generate:sqlite"
  }
}

in which we use npm-specifiers to invoke the drizzle-kit cli tool to generate sql migration files. If this were working correctly (and didn't produce the error above out of its esbuild dependency) I would expect the tool to successfully generate files in the./drizzle directory, at which point running deno task dev would work, though that part is not necessarily the subject of this issue report.

Notes

I am running on Linux (Pop_OS! 22.04) and here is the output of deno --version

deno --version                                                                                                                                                                                                                                                                      
deno 1.35.1 (release, x86_64-unknown-linux-gnu)
v8 11.6.189.7
typescript 5.1.6

[adamgreg]

I also want to use drizzle-kit in my project and have the same issue.

As an alternative, I've tried importing it from https://esm.sh/[email protected], but get a confusing HTTP 500 error: "[esm.sh] could not resolve "drizzle-kit".

esbuild works in Deno if imported from https://deno.land/x/. Is there any way to force npm:drizzle-kit to import it from there instead?

[DMeechan]

@adamgreg I've had the same issue trying to use drizzle-kit with Deno (deno 1.36.4).

I get a mysterious 500 when using the esm.sh URI:

deno run -A https://esm.sh/[email protected] introspect:pg --verbose

error: Import 'https://esm.sh/[email protected]' failed: 500 Internal Server Error

And the same worker.unref is not a function error when using the npm URI:

deno run -A npm:drizzle-kit introspect:pg

drizzle-kit: v0.19.12
drizzle-orm: v0.28.5

No config path provided, using default 'drizzle.config.ts'
Reading config file '/home/project/drizzle.config.ts'
error: Uncaught (in promise) TypeError: worker.unref is not a function
    at <anonymous> (file:///home/project/node_modules/.deno/[email protected]/node_modules/esbuild/lib/main.js:2302:10)

As a workaround, I'm probably going to generate migrations by running drizzle-kit on Node (while keeping the rest of the project on Deno).

[adamgreg]

@DMeechan Same here - I've been running migrations using a separate Node project, and copying the generated files across. It's working ok for me, but I'd love to be able to do it within my Deno project.

Probably worth mentioning this Drizzle issue here: https://github.com/drizzle-team/drizzle-orm/issues/252

[bjoernwuest]

Quite similar here.

drizzle.config.ts:

import type { Config } from "drizzle-kit";

export default {
    schema: "./**/*.schema.ts",
    out: "./drizzle-migration",
    verbose: true
} satisfies Config;

deno --version

deno 1.37.0 (release, x86_64-pc-windows-msvc)
v8 11.8.172.3
typescript 5.2.2

deno run -A npm:drizzle-kit generate:pg

drizzle-kit: v0.19.13
drizzle-orm: v0.28.6

No config path provided, using default path
Reading config file '...\drizzle.config.ts'
error: Uncaught (in worker "$DENO_STD_NODE_WORKER_THREAD") SyntaxError: Octal escape sequences are not allowed in strict mode.
    at <anonymous> (data:text/javascript,(async function() {c......\lib\main.js");})();:1:147)
error: Uncaught (in promise) TypeError: worker.unref is not a function
    at <anonymous> (file:///.../node_modules/.deno/[email protected]/node_modules/esbuild/lib/main.js:2306:10)

[patrickalima98]

Note Demo / reproduction repository: https://github.com/andrewbrey/deno_npm_compat_drizzle-orm

When attempting to use the drizzle-orm (https://orm.drizzle.team) tool called drizzle-kit to generate sql schema migrations from TypeScript model definitions via npm-specifiers in Deno, I encountered the following error:

error: Uncaught TypeError: worker.unref is not a function
    at <anonymous> (file:///home/andrew/.cache/deno/npm/registry.npmjs.org/esbuild/0.18.12/lib/main.js:2295:10)

The linked repository ( https://github.com/andrewbrey/deno_npm_compat_drizzle-orm ) contains a minimal setup needed to test out if drizzle-orm works with Deno npm-specifier based compatibility. The main point of interest is actually the migrate task defined in the deno.jsonc:

// deno.jsonc

{
  "tasks": {
    ...
    "migrate": "deno run -A npm:drizzle-kit generate:sqlite"
  }
}

in which we use npm-specifiers to invoke the drizzle-kit cli tool to generate sql migration files. If this were working correctly (and didn't produce the error above out of its esbuild dependency) I would expect the tool to successfully generate files in the./drizzle directory, at which point running deno task dev would work, though that part is not necessarily the subject of this issue report.

Notes

I am running on Linux (Pop_OS! 22.04) and here is the output of deno --version

deno --version                                                                                                                                                                                                                                                                      
deno 1.35.1 (release, x86_64-unknown-linux-gnu)
v8 11.6.189.7
typescript 5.1.6

The same problem to me :(

[foobarfighter]

I hit this issue as well.

[kwhinnery]

It looks like we are missing an implementation of this method on workers:

https://nodejs.org/api/worker_threads.html#workerunref

[kwhinnery]

Some possible workarounds for folks wanting to use Drizzle schemas in Deno immediately: https://gist.github.com/kwhinnery/fc4f4d74fc71c7001ae0881447147940

[davepar]

Thank you for that gist @kwhinnery. I was going around in circles trying to make Drizzle Kit work from Deno, and it just wasn't happening. Two additional steps I needed, npm init to create the package.json since that normally isn't needed for a Deno project, and I had to also install the pg package in order to use Drizzle Kit Studio, so npm i -D pg.

[waptik]

Some possible workarounds for folks wanting to use Drizzle schemas in Deno immediately: https://gist.github.com/kwhinnery/fc4f4d74fc71c7001ae0881447147940

I tried following your instructions but hit a roadblock. I'm using sqlite

UPDATE:

It's working now. I forgot that my schema index file is mod.ts instead of index.ts hence why the error message included Err: no schema files files found... when i used yarn drizzle-kit ...

[windchime-yk]

I have tried worker.unref since it seems to have been implemented, but it stops the process without giving any errors.

$ deno run -A npm:drizzle-kit generate:mysql --schema=./db/schema.ts
drizzle-kit: v0.19.12
drizzle-orm: v0.27.2

Reading schema files:
/home/windchime-yk/dev/private/cinelog/db/schema.ts

Notes

Deno version

$ deno --version
deno 1.41.2 (release, x86_64-unknown-linux-gnu)
v8 12.1.285.27
typescript 5.3.3

[bartlomieju]

@windchime-yk does it work correctly if you use "Bring Your Own Node_Modules"? (https://deno.com/blog/v1.38#nodejs-compatibility-improvements)

[windchime-yk]

@bartlomieju Thank you for the suggestion: This is my first time using "Bring Your Own Node_Modules", so I may not be entirely familiar with it yet. However, I attempted to use it on the project I used for verification. Unfortunately, like my initial attempt, the process halted unexpectedly midway through. https://github.com/windchime-yk/cinelog/pull/67/files

$ deno run -A npm:drizzle-kit generate:mysql --schema=./db/schema.ts
drizzle-kit: v0.20.14
drizzle-orm: v0.30.1

Reading schema files:
/home/windchime-yk/dev/private/cinelog/db/schema.ts

[adamgreg]

Update: with Deno 1.42.1 I can successfully run deno run -A npm:drizzle-kit generate:pg and deno run -A npm:drizzle-kit drop. The --unstable-byonm flag is not needed, but I do need a local node_modules folder containing drizzle-orm (or to use the --node-modules-dir flag when running). Without it, I get Uncaught Error: Cannot find module 'drizzle-orm/pg-core'.

Drizzle Studio still doesn't work (it looks like an incoming message from the node http compat layer is missing the rawHeaders property for some reason).

[igorbrasileiro]

Update: with Deno 1.42.1 I can successfully run deno run -A npm:drizzle-kit generate:pg and deno run -A npm:drizzle-kit drop. The --unstable-byonm flag is not needed, but I do need a local node_modules folder containing drizzle-orm (or to use the --node-modules-dir flag when running). Without it, I get Uncaught Error: Cannot find module 'drizzle-orm/pg-core'.

Drizzle Studio still doesn't work (it looks like an incoming message from the node http compat layer is missing the rawHeaders property for some reason).

I'm trying to run deno run -A npm:drizzle-kit generate with sqlite+turso, but without success. That's the stacktrace of the error:

drizzle-kit: v0.21.2
drizzle-orm: v0.30.10

No config path provided, using default 'drizzle.config.ts'
Reading config file '.../path/here/drizzle.config.ts'
error: Uncaught (in promise) Error: Cannot find module 'drizzle-kit'

[guy-borderless]

as it stands, it's best to globally install via npm i -g drizzle-orm drizzle-kit

[adamgreg]

Hi @igorbrasileiro, sorry for the slow reply! The key is that you need a node_modules directory, and to have drizzle-kit in there already. I do:

deno cache --node-modules-dir drizzle-kit
deno run -A --node-modules-dir drizzle-orm generate

That does the trick for generate and drop. studio only runs with Node, last time I checked. I haven't tried the other commands, but I'd expect them to work.

[igorbrasileiro]

@adamgreg I tried to run, but still facing issues with npm dependencies.

Task db:setup:deps deno cache --node-modules-dir npm:drizzle-orm npm:drizzle-orm/sqlite-core npm:drizzle-kit
Task db:setup:generate deno run -A npm:drizzle-kit generate
drizzle-kit: v0.21.2
drizzle-orm: v0.30.10

No config path provided, using default 'drizzle.config.ts'
Reading config file '/Users/igorbrasileiro/dev/deco/bras-teste/drizzle.config.ts'
Error: Cannot find module 'npm:drizzle-orm/sqlite-core'
Require stack:
- <project-path>/bras-teste/db/schema.ts
- <SO user path>/Library/Caches/deno/npm/registry.npmjs.org/drizzle-kit/0.21.2/bin.cjs

[bartlomieju]

@igorbrasileiro looks like drizzle-orm depend on native addons, which are currently not supported in Deno. You can work around it in the meantime by installing using npm/yarn/pnpm.

[igorbrasileiro]

@igorbrasileiro looks like drizzle-orm depend on native addons, which are currently not supported in Deno. You can work around it in the meantime by installing using npm/yarn/pnpm.

Thank you for the help @bartlomieju . Does deno have plans to support it?

[bartlomieju]

Yes, keep an eye on https://github.com/denoland/deno/issues/16164 for the progress.

[adamgreg]

Thanks for the link to the relevant issue @bartlomieju!

@igorbrasileiro in your log snippet, it looks the --node-modules-dir argument is missing from the deno run command. Worth a try? I haven't tried with sqlite, just Postgres.

[igorbrasileiro]

@adamgreg

Task db:setup deno task db:setup:deps && deno task db:setup:generate && deno task db:migrate
Task db:setup:deps deno cache --node-modules-dir npm:drizzle-orm npm:drizzle-orm/sqlite-core npm:drizzle-kit
Task db:setup:generate deno run -A --node-modules-dir npm:drizzle-kit generate
drizzle-kit: v0.21.2
drizzle-orm: v0.30.10

No config path provided, using default 'drizzle.config.ts'
Reading config file '/Users/igorbrasileiro/dev/deco/bras-teste/drizzle.config.ts'
Error: Cannot find module 'npm:drizzle-orm/sqlite-core'
Require stack:
- <...>/bras-teste/db/schema.ts
- <...>/bras-teste/node_modules/.deno/[email protected]/node_modules/drizzle-kit/bin.cjs

[bartlomieju]

Folks I highly recommend running with DENO_FUTURE=1 env var - which sort of enables "Deno 2 mode". You can then install drizzle using npm/yarn/pnpm and run program using deno run -A npm:drizzle-kit generate. This should solve the Cannot find module ... errors you are seeing.

[adamgreg]

@igorbrasileiro I'm more or less out of ideas, except to say you that if you have an npm: import in the config module, maybe that's the problem. You could try losing it if it's just for defineConfig, or use an import map in deno.jsonc so you can import without the npm: prefix.

I haven't tried DENO_FUTURE=1 myself yet, but will definitely give it a go.

[miguelrk]

Thanks , but @bartlomieju to my understanding, Deploy currently does not support running with DENO_FUTURE right?

[igorbrasileiro]

@bartlomieju and @adamgreg really thank you. The DENO_FUTURE env var and remove all imports with this syntax npm:<package-name> on .ts files helped to run. I let my setup below to help others

Removed all imports with npm: from .ts files on deno project and put it on import_map, like this:

    "drizzle-orm": "npm:[email protected]",
    "drizzle-orm/sqlite-core": "npm:[email protected]/sqlite-core",
    "drizzle-kit": "npm:[email protected]",
    "drizzle-kit/": "npm:/[email protected]/",
    "@libsql/client": "npm:@libsql/[email protected]"

I maintained these deno tasks below

    "db:setup:deps": "deno cache -r --node-modules-dir npm:drizzle-orm npm:drizzle-orm/sqlite-core npm:drizzle-kit npm:@libsql/client",
    "db:setup:generate": "deno run -A --node-modules-dir npm:drizzle-kit generate",
    "db:setup": "deno task db:setup:deps && deno task db:setup:generate && deno task db:migrate",
    "db:migrate": "deno run -A --node-modules-dir npm:drizzle-kit migrate"

Then, tu run, i changed the command adding DENO_FUTURE=1 env var.

DENO_FUTURE=1 deno task db:setup

[ClaudiuCeia]

@bartlomieju and @adamgreg really thank you. The DENO_FUTURE env var and remove all imports with this syntax npm:<package-name> on .ts files helped to run. I let my setup below to help others

Removed all imports with npm: from .ts files on deno project and put it on import_map, like this:

    "drizzle-orm": "npm:[email protected]",
    "drizzle-orm/sqlite-core": "npm:[email protected]/sqlite-core",
    "drizzle-kit": "npm:[email protected]",
    "drizzle-kit/": "npm:/[email protected]/",
    "@libsql/client": "npm:@libsql/[email protected]"

I maintained these deno tasks below

    "db:setup:deps": "deno cache -r --node-modules-dir npm:drizzle-orm npm:drizzle-orm/sqlite-core npm:drizzle-kit npm:@libsql/client",
    "db:setup:generate": "deno run -A --node-modules-dir npm:drizzle-kit generate",
    "db:setup": "deno task db:setup:deps && deno task db:setup:generate && deno task db:migrate",
    "db:migrate": "deno run -A --node-modules-dir npm:drizzle-kit migrate"

Then, tu run, i changed the command adding DENO_FUTURE=1 env var.

DENO_FUTURE=1 deno task db:setup

Thanks man, this worked for me too

[elliotnash]

@bartlomieju and @adamgreg really thank you. The DENO_FUTURE env var and remove all imports with this syntax npm:<package-name> on .ts files helped to run. I let my setup below to help others

Removed all imports with npm: from .ts files on deno project and put it on import_map, like this:

    "drizzle-orm": "npm:[email protected]",
    "drizzle-orm/sqlite-core": "npm:[email protected]/sqlite-core",
    "drizzle-kit": "npm:[email protected]",
    "drizzle-kit/": "npm:/[email protected]/",
    "@libsql/client": "npm:@libsql/[email protected]"

I maintained these deno tasks below

    "db:setup:deps": "deno cache -r --node-modules-dir npm:drizzle-orm npm:drizzle-orm/sqlite-core npm:drizzle-kit npm:@libsql/client",
    "db:setup:generate": "deno run -A --node-modules-dir npm:drizzle-kit generate",
    "db:setup": "deno task db:setup:deps && deno task db:setup:generate && deno task db:migrate",
    "db:migrate": "deno run -A --node-modules-dir npm:drizzle-kit migrate"

Then, tu run, i changed the command adding DENO_FUTURE=1 env var.

DENO_FUTURE=1 deno task db:setup

My drizzle.config.ts loads the database credentials from a config.jsonc file using @std/jsonc. When using this setup, any npm packages import correctly, but any non npm package fails to import

error: Uncaught (in promise) Error: Cannot find module 'jsr:@std/jsonc@^1.0.0-rc.2'

Does anyone have a suggestion on how to use jsr libraries from within the the drizzle.config.ts?

[adamgreg]

@elliotnash Just to be clear, do you mean you're running drizzle-kit using Deno, and when it hits this JSR import it fails?

If so, I guess it's a more general issue of dynamic imports from node modules being unable to import from JSR. Do you have the same problem with https imports? If not, you could import an older version of @std/jsonc from https://deno.land/x.