Transactions not working in Node 20

[jakeleventhal]

trafficstars

I previously had no issues until I tried to upgrade from node 18.19.1 to 20.12.2. When making a parallel insert in a transaction, it seems to lose sight of the transaction scope when using neon serverless.

Below is the drizzle db object I am exporting. When I hardcode to always use the postgres-js version, it works fine. If I hardcode to use the neon-serverless version, it fails.

import { neonConfig, Pool } from '@neondatabase/serverless';
import { drizzle as neonDrizzle, NeonDatabase } from 'drizzle-orm/neon-serverless';
import { drizzle } from 'drizzle-orm/postgres-js';
import postgres from 'postgres';
import ws from 'ws';

import * as schema from './schema';

neonConfig.webSocketConstructor = ws;

const drizzleClient = (datasourceUrl: string | undefined) => {
  const connectionString = datasourceUrl || process.env.DATABASE_URL;

  const client = () => {
    if (process.env.SERVERLESS_DRIVER) {
      const pool = new Pool({ connectionString });
      return neonDrizzle(pool, { schema });
    }

    return drizzle(postgres(connectionString!), { schema }) as unknown as NeonDatabase<typeof schema>;
  };

  if (!connectionString) {
    return null as any as ReturnType<typeof client>;
  }

  return client();
};

declare global {
  // eslint-disable-next-line vars-on-top, no-var
  var localDrizzle: ReturnType<typeof drizzleClient>;
}

export const getDrizzleClient = (url?: string): ReturnType<typeof drizzleClient> => {
  if (process.env.SERVER || url) {
    return drizzleClient(url);
  }

  if (!global.localDrizzle) {
    global.localDrizzle = drizzleClient(url);
  }
  return global.localDrizzle;
};

const db = getDrizzleClient();
export default db;

export * from './appData';
export * from './schema';
export * from './types';

Below is my application code

const user = await db.transaction(async (tx) => {
  const [newUser] = await tx
    .insert(users)
    .values({
      email,
      emailVerificationEmail: email,
      id: userId,
      password: Utils.getHash(password),
      stripeCustomerId: await StripeUtils.createStripeCustomer(userId, email),
      ...(!process.env.SERVER && {
        emailVerificationEmail: null,
        emailVerificationId: null
      })
    })
    .returning();

  const [{ id: addressId }] = await tx.insert(addresses).values({ userId }).returning();

  const brandingId = crypto.randomUUID();
  await Promise.all([
    tx.insert(brandings).values({ id: brandingId, userId }).execute(),
    tx.insert(userSettings).values({ addressId, userId }).execute()
  ]);
  await tx.insert(packageInserts).values({ brandingId }).execute();

  return newUser;
});

If I remove the Promise.all and instead do each of the inserts one at a time, I do not get an error. Below is the error I end up getting:

Apr 23, 4:18:05PM error: error: insert or update on table "UserSettings" violates foreign key constraint "UserSettings_userId_User_id_fk"
    at eval (webpack-internal:///(api)/../../../node_modules/.pnpm/@[email protected]/node_modules/@neondatabase/serverless/index.mjs:1355:74)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 1)
    at async eval (webpack-internal:///(api)/./src/pages/api/account/sign-up.tsx:63:9)
    at async NeonSession.transaction (file:///Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]_@verce_bkgk4kkqtyvt7agio77yci4idy/node_modules/drizzle-orm/neon-serverless/session.js:96:22)
    at async __WEBPACK_DEFAULT_EXPORT__.requireUserAuth (webpack-internal:///(api)/./src/pages/api/account/sign-up.tsx:42:18)
    at async eval (webpack-internal:///(api)/../../../packages/riptech/utils/src/errors/client/clientErrorWrapper.ts:50:13)
    at async K (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected][email protected]/node_modules/next/dist/compiled/next-server/pages-api.runtime.dev.js:21:2871)
    at async U.render (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected][email protected]/node_modules/next/dist/compiled/next-server/pages-api.runtime.dev.js:21:3955)
    at async DevServer.runApi (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected][email protected]/node_modules/next/dist/server/next-server.js:600:9)
    at async NextNodeServer.handleCatchallRenderRequest (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected][email protected]/node_modules/next/dist/server/next-server.js:269:37)
    at async DevServer.handleRequestImpl (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected][email protected]/node_modules/next/dist/server/base-server.js:816:17)
    at async /Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected][email protected]/node_modules/next/dist/server/dev/next-dev-server.js:339:20
    at async Span.traceAsyncFn (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected][email protected]/node_modules/next/dist/trace/trace.js:154:20)
    at async DevServer.handleRequest (/Users/jakeleventhal/Code/rip-technologies/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected][email protected]/node_modules/next/dist/server/dev/next-dev-server.js:336:24) 

Schema shown below for extra info:

export const users = pgTable(
  'User',
  {
    company: text('company').default('').notNull(),
    email: text('email').notNull(),
    emailVerificationEmail: text('emailVerificationEmail'),
    emailVerificationId: text('emailVerificationId').$defaultFn(() => sql`gen_random_uuid()::text`),
    firstName: text('firstName').default('').notNull(),
    id: text('id')
      .primaryKey()
      .$defaultFn(() => sql`gen_random_uuid()::text`)
      .notNull(),
    joinedDate: customTimestampWithTimezone('joinedDate')
      .default(sql`now()`)
      .notNull(),
    lastLoginDate: customTimestampWithTimezone('lastLoginDate')
      .default(sql`now()`)
      .notNull(),
    lastName: text('lastName').default('').notNull(),
    logins: integer('logins').default(1).notNull(),
    password: text('password').notNull(),
    passwordResetId: text('passwordResetId'),
    paymentFailureDate: customTimestampWithTimezone('paymentFailureDate'),
    phone: text('phone').default('').notNull(),
    sampleCreditBalance: doublePrecision('sampleCreditBalance').default(100).notNull(),
    stripeCustomerId: text('stripeCustomerId').notNull()
  },
  (table) => ({
    emailKey: uniqueIndex('User_email_key').on(table.email),
    stripeCustomerIdKey: uniqueIndex('User_stripeCustomerId_key').on(table.stripeCustomerId)
  })
);

export const userSettings = pgTable(
  'UserSettings',
  {
    addressId: text('addressId')
      .notNull()
      .references(() => addresses.id, { onDelete: 'restrict', onUpdate: 'cascade' }),
    canadianBusinessNumber: text('canadianBusinessNumber').default('').notNull(),
    displayCurrency: DisplayCurrencyEnum('displayCurrency').default('USD').notNull(),
    id: text('id')
      .primaryKey()
      .$defaultFn(() => sql`gen_random_uuid()::text`)
      .notNull(),
    // The ids of the user's invalid cards that failed to process payments.
    invalidCardIds: text('invalidCardIds')
      .array()
      .default(sql`'{}'`) // Empty array
      .notNull(),
    onboardingStep: OnboardingStepEnum('onboardingStep').default('PersonalDetails'),
    primaryCardId: text('primaryCardId'),
    themeMode: ThemeModeEnum('themeMode').default('light').notNull(),
    userId: text('userId')
      .notNull()
      .references(() => users.id, { onDelete: 'cascade', onUpdate: 'cascade' })
  },
  (table) => ({
    addressIdKey: uniqueIndex('UserSettings_addressId_key').on(table.addressId),
    userIdKey: uniqueIndex('UserSettings_userId_key').on(table.userId)
  })
);

[jakeleventhal]

Cross referencing: https://github.com/drizzle-team/drizzle-orm/issues/2200

[jawj]

Is it reproducibly the case that Node 18 always works and Node 20 always doesn't? If so, I think it's more likely an issue on the Drizzle side.

@conradludgate Is it otherwise possible that this could be caused by any recent changes to connection pooling?

[conradludgate]

We've seen several bugs related to node 20 being reported in our discord (Specifically ETIMEDOUT errors). We've made no changes to connection pooling recently but we did upgrade our HTTP server library. I would be surprised if node 20 could not connect to it though. I'll try and find some time to see if I can reproduce to debug further.

In this case, it looks like you are connecting fine, so that seems like a different circumstance.

[jakeleventhal]

@jawj Yes, I have been using 18 for several months. Just tried to switch to 20 and got roadblocked here. Cannot get it work unless I remove any parallel actions in the the transaction.

[jakeleventhal]

Confirmed this actually wasn't node 20. The real problem is in pnpm it seems https://github.com/pnpm/pnpm/issues/8039

[jakeleventhal]

@jawj i would like to reopen this. I am finding that this issue is only occurring when I use a Neon driver in drizzle. and only when i upgrade from pnpm 8→9

[jawj]

OK. I think it's more likely either pnpm or drizzle, but if you can provide a minimal reproduction in a repo I'll be happy to take a look.

[jakeleventhal]

I'm going to be switching off of neon actually. so i think we can close this