[BUG]: NodeJS Buffer used in SQLiteBigInt.mapToDriverValue

[plancky]

What version of drizzle-orm are you using?

0.30.3

What version of drizzle-kit are you using?

1.0.31

Describe the Bug

I am running a drizzle client based on drizzle-orm/d1 and schema generated from datatypes defined in drizzle-orm/sqlite-core in a workerd runtime.

Writing to a column of type blob(mode: bigInt) gives back this error ReferenceError: Buffer is not defined.

I am assuming this is because I am running the orm in workerd which doesn't implement the Buffer API.

Expected behavior

Should work on workerd out of the box.

Environment & setup

No response

[Soviut]

I have the same issue but with a json blob.

I tried adding a node compatibility flag to wrangler.toml but it did not seem to work.

compatibility_flags = [ "nodejs_compat" ]

[Soviut]

Also worth noting, there is a Buffer available in workers but it needs to be imported

import { Buffer } from 'node:buffer'

https://developers.cloudflare.com/workers/runtime-apis/nodejs/buffer/

[sdarnell]

I'd suggest Drizzle switch to use Uint8Array instead of Buffer as this is near functional equivalent, and more portable. Buffer is also a subclass of Uint8Array, so from a type perspective existing callers can still pass Buffers. Further background info: https://sindresorhus.com/blog/goodbye-nodejs-buffer

I have an different, but related issue with Buffer that doesn't pass through Miniflare to D1, and switching to Uint8Array would help there too.

[jack-davies]

I also bumped into this, and was able to work around the issue by first enabling compatibility_flags = [ "nodejs_compat" ] in wrangler.toml, then by adding the following to db/schema.ts:

import { Buffer } from "node:buffer";
globalThis.Buffer = Buffer;

Which ensures that when Buffer is called it is defined globally.

A proper fix in drizzle-orm would be to add just the import:

import { Buffer } from "node:buffer";

to any modules that use Buffer such as blob.ts. This is best practice:

While the Buffer class is available within the global scope, it is still recommended to explicitly reference it via an import or require statement.^1

[Soviut]

@jack-davies CloudFlare also recently announced further node compatibility, most likely specifically for this situation using the node_compat_v2 flag, or the old node_compat with a compatibility_date of 2024-09-23 or later.

This will allow things like Buffer to be used without the node: prefix on import. Meaning libraries that depend on it won't have to be altered.

https://developers.cloudflare.com/workers/configuration/compatibility-dates/#nodejs-compatibility-flag

You can opt into improved Node.js compatibility by using nodejs_compat_v2 instead of nodejs_compat. This provides the functionality of nodejs_compat, but additionally you can import Node.js modules without the node: prefix and use polyfilled Node.js modules and globals that are not available with nodejs_compat.

On September 23, 2024, nodejs_compat will use the improved Node.js compatibility currently enabled with nodejs_compat_v2. This will require updating your compatability_date to 2024-09-23 or later.