Can't build with knex module

[shadoworion]

What version of Bun is running?

1.0.20+09d51486e

What platform is your computer?

Darwin 23.2.0 arm64 arm

What steps can reproduce the bug?

{
  "name": "backend",
  "module": "src/server.ts",
  "type": "module",
  "scripts": {
    "dev": "bun gen && bun ./src/server.ts",
    "build": "rm -rf dist && bun build --target=bun ./src/server.ts --outfile=./dist/server.js",
    "gen": "bun prisma generate",
    "migrate": "bun prisma migrate dev"
  },
  "devDependencies": {
    "@types/jsonwebtoken": "^9.0.5",
    "@types/uuid": "^9.0.7",
    "@types/crypto-js": "^4.2.1",
    "@types/mime-types": "^2.1.4",
    "@types/node-schedule": "^2.1.5",
    "@types/pg": "^8.10.9",
    "@types/bun": "^1.0.0",
    "prisma": "^5.7.1"
  },
  "peerDependencies": {
    "typescript": "^5.3.3"
  },
  "dependencies": {
    "@aws-sdk/client-ecs": "^3.481.0",
    "@aws-sdk/client-s3": "^3.481.0",
    "@aws-sdk/s3-request-presigner": "^3.481.0",
    "@bloxtax/extras": "^1.0.6",
    "@clerk/clerk-sdk-node": "^4.13.4",
    "@pothos/core": "^3.41.0",
    "@pothos/plugin-dataloader": "^3.17.2",
    "@pothos/plugin-prisma": "^3.63.0",
    "@prisma/client": "^5.7.1",
    "@sendgrid/mail": "^8.1.0",
    "@solana/web3.js": "^1.87.6",
    "@whatwg-node/server-plugin-cookies": "^1.0.2",
    "axios": "^1.6.3",
    "crypto-js": "^4.2.0",
    "dataloader": "^2.2.2",
    "dayjs": "^1.11.10",
    "firebase-admin": "^12.0.0",
    "form-data": "^4.0.0",
    "graphql": "^16.8.1",
    "graphql-middleware": "^6.1.35",
    "graphql-request": "^6.1.0",
    "graphql-scalars": "^1.22.4",
    "graphql-shield": "^7.6.5",
    "graphql-yoga": "^5.1.0",
    "jsonwebtoken": "^9.0.2",
    "knex": "^3.1.0",
    "libphonenumber-js": "^1.10.53",
    "mime-types": "^2.1.35",
    "node-object-hash": "^3.0.0",
    "node-schedule": "^2.1.1",
    "pg": "^8.11.3",
    "query-string": "^8.1.0",
    "redis": "^4.6.12",
    "uuid": "^9.0.1",
    "web3-validator": "^2.0.3",
    "xlsx": "https://cdn.sheetjs.com/xlsx-0.20.1/xlsx-0.20.1.tgz"
  },
  "engines": {
    "node": ">=18"
  }
}

bun run build

What is the expected behavior?

Success build

What do you see instead?

$ rm -rf dist && bun build --target=bun ./src/server.ts --outfile=./dist/server.js
7 |     return require('better-sqlite3');
                       ^
error: Could not resolve: "better-sqlite3". Maybe you need to "bun install"?
    at /backend/node_modules/knex/lib/dialects/better-sqlite3/index.js:7:20

15 |     return require('mysql2');
                        ^
error: Could not resolve: "mysql2". Maybe you need to "bun install"?
    at /backend/node_modules/knex/lib/dialects/mysql2/index.js:15:20

24 |     return require('mysql');
                        ^
error: Could not resolve: "mysql". Maybe you need to "bun install"?
    at /backend/node_modules/knex/lib/dialects/mysql/index.js:24:20

94 |     const tds = require('tedious');
                             ^
error: Could not resolve: "tedious". Maybe you need to "bun install"?
    at /backend/node_modules/knex/lib/dialects/mssql/index.js:94:25

44 |     const oracledb = require('oracledb');
                                  ^
error: Could not resolve: "oracledb". Maybe you need to "bun install"?
    at /backend/node_modules/knex/lib/dialects/oracledb/index.js:44:30

42 |     return require('sqlite3');
                        ^
error: Could not resolve: "sqlite3". Maybe you need to "bun install"?
    at /backend/node_modules/knex/lib/dialects/sqlite3/index.js:42:20

192 |       : require('pg-query-stream');
                      ^
error: Could not resolve: "pg-query-stream". Maybe you need to "bun install"?
    at /backend/node_modules/knex/lib/dialects/postgres/index.js:192:17
error: script "build" exited with code 1

Additional information

This is an optional drivers of knex. I don't need all of them.

[Electroid]

Did you run bun install?

[shadoworion]

Did you run bun install?

Sure

[TiBianMod]

add this deps on your local devDependencies

{
    "aws-sdk": "^2.1526.0",
    "better-sqlite3": "^9.2.2",
    "mock-aws-s3": "^4.0.2",
    "mysql": "^2.18.1",
    "mysql2": "^3.6.5",
    "nock": "^13.4.0",
    "oracledb": "^6.3.0",
    "pg-query-stream": "^4.5.3",
    "sqlite3": "^5.1.6",
    "tedious": "^16.6.1"
}

this are devDependencies on knex, but missing to you locally

bun install bun run build

[shadoworion]

add this deps on your local devDependencies

{
    "aws-sdk": "^2.1526.0",
    "better-sqlite3": "^9.2.2",
    "mock-aws-s3": "^4.0.2",
    "mysql": "^2.18.1",
    "mysql2": "^3.6.5",
    "nock": "^13.4.0",
    "oracledb": "^6.3.0",
    "pg-query-stream": "^4.5.3",
    "sqlite3": "^5.1.6",
    "tedious": "^16.6.1"
}

this are devDependencies on knex, but missing to you locally

bun install bun run build

I understand that it will fix my problem, but it is bug. It may work without these packages.

[shadoworion]

Knex requires you to install only drivers you need, not always all drivers. Build works without these packages in ts-node. So it may also work with bun.

[TiBianMod]

Knex requires you to install only drivers you need, not always all drivers. Build works without these packages in ts-node. So it may also work with bun.

this is good news that is working with ts-node, because I have the same issue, the final bundler is 29+ mb

[djamez]

A work-around that I use involves delving into node-modules - which is always a bad idea. If you want to go this route though, open /node-modules/knex/lib/dialects/index.js and comment out the drivers you do not want in const dbNameToDialectLoader. Notice - if you want to build using MySQL2, you must also install MySQL.

[Michael-Liendo]

add this deps on your local devDependencies

{
    "aws-sdk": "^2.1526.0",
    "better-sqlite3": "^9.2.2",
    "mock-aws-s3": "^4.0.2",
    "mysql": "^2.18.1",
    "mysql2": "^3.6.5",
    "nock": "^13.4.0",
    "oracledb": "^6.3.0",
    "pg-query-stream": "^4.5.3",
    "sqlite3": "^5.1.6",
    "tedious": "^16.6.1"
}

this are devDependencies on knex, but missing to you locally

bun install bun run build

I understand that it will fix my problem, but it is bug. It may work without these packages.

Same here! that works for me, but it's weird to have to do that.

[guillaumemeyer]

The workaround works for me too. Anybody found another solution?

[paperclover]

the error message here is very poor. the correct solution is to pass --external with all of the unused packages. this will silence the bundler resolution error. for example, if you are using sqlite3, you pass --external oracledb --external mysql --external tedious --external mysql2 --external better-sqlite3 --external pg-query-stream (excluding --external sqlite3).

these errors originate because knex has many drivers, which are all bundled together. they use require to load them on-demand.

example:

bundling attempts to combine all dependencies into one file, by design. the error is correct that it cannot resolve this. by using --external, the bundler never even attempts to bundle these, and will just emit runtime require/await import calls. that looks like this now:

if you hit this path at runtime, it will call out to the actual require, look in node_modules, and then fail then (with --target=browser it will look for a global variable named require and call that).

esbuild has a great error message for this. we should adopt this in the future, but that is another issue.

[paperclover]

i should also mention that another solution to this, if bundling node_modules is not a concern, is to use the recently added --packages=external flag. This marks every node_module package as external, and only bundling application code.

[vinhqx]

@guillaumemeyer I found an option named --external (which shorcuted -e) on bun build command.

Solution:

bun build <filename> -e tedious -e pg -e pg-query-stream -e oracledb -e mysql -e better-sqlite3 -e mysql2