[BUG]: Cannot use a dynamic select query as a subquery anymore (regression bug)

[AndrewJo]

trafficstars

Report hasn't been filed before.

• [x] I have verified that the bug I'm about to report hasn't been filed before.

What version of drizzle-orm are you using?

0.39.1

What version of drizzle-kit are you using?

0.30.4

Other packages

No response

Describe the Bug

Steps to reproduce

Create a dynamic select query and call .as to pass it in as a subquery:

const subquery = drizzle
  .select({ id: schema.users.id })
  .from(schema.users)
  .$dynamic();

const query = drizzle
  .select({ count: sql<number>`cast(count(*) as int)` })
  .from(subquery.as('__subquery__'))
//      ^ Argument of type SubqueryWithSelection<T["_"]["selectedFields"], "__subquery__"> is not assignable to parameter of type 'TableLikeHasEmptySelection<SubqueryWithSelection<T["_"]["selectedFields"], "__subquery__">> extends true ? DrizzleTypeError<...> : SubqueryWithSelection<...>'.ts(2345)

This results in a TypeScript compile error:

Argument of type SubqueryWithSelection<T["_"]["selectedFields"], "__subquery__"> is not assignable to parameter of type 'TableLikeHasEmptySelection<SubqueryWithSelection<T["_"]["selectedFields"], "__subquery__">> extends true ? DrizzleTypeError<...> : SubqueryWithSelection<...>'.ts(2345)

This previously worked before and seems to have regressed when this was merged in:

• #3976

I still need to investigate if it's also broken in runtime. If not, the workaround is probably to just cast it as any for now.

[wottpal]

Same issue here!

[CorySpeed]

Same issue here.

I'm using a 'this.table' within a class which extends the AnyPgTable typing and getting the following: Argument of type 'T' is not assignable to parameter of type 'TableLikeHasEmptySelection<T> extends true ? DrizzleTypeError<"Cannot reference a data-modifying statement subquery if it doesn't contain a returning clause"> : T'.

I've downgraded back to 0.38.0 at the moment

[doug-martin]

Same issue as @CorySpeed

[Meess]

Same issue for all my table wrapper functions:

import db from '../db'
import { type Table, type SQL, } from 'drizzle-orm'

function queryTable<T extends Table>(table: T, whereClause?: SQL<unknown>) {
    let query = db.select().from(table)
}

from(table) will give the errror:

Argument of type 'T' is not assignable to parameter of type 'TableLikeHasEmptySelection<T> extends true ? DrizzleTypeError<"Cannot reference a data-modifying statement subquery if it doesn't contain a returning clause"> : T'. Type 'Table<TableConfig<Column<any, object, object>>>' is not assignable to type 'TableLikeHasEmptySelection<T> extends true ? DrizzleTypeError<"Cannot reference a data-modifying statement subquery if it doesn't contain a returning clause"> : T'

For now downgraded to 0.38.4, as this type bug was introduced in 0.39.0. Curious to hear how other people solved typings for their wrapper functions

[wottpal]

Curious to hear how other people solved typings for their wrapper functions.

as any 😭

[CorySpeed]

For now downgraded to 0.38.4, as this type bug was introduced in 0.39.0. Curious to hear how other people solved typings for their wrapper functions

I've not been able to upgrade yet still due to this issue.

[Navigator12]

Started a new project but had to proceed with a downgraded version instead.

[CorySpeed]

Is there any update on this as of yet?

[AndriiSherman]

@Sukairo-02 just took it and will update you once he find the issue and a solution

[rasel-stacklearner]

Is there any update on this?

[Sukairo-02]

I've been trying to reproduce this issue on both mentioned and latest versions of orm - to no avail. To proceed, I'm going to need either reproduction repo, or more details on issue, such as table's configuration and project's tsconfig, and source of drizzle element in your code.

[Meess]

I've been trying to reproduce this issue on both mentioned and latest versions of orm - to no avail. To proceed, I'm going to need either reproduction repo, or more details on issue, such as table's configuration and project's tsconfig, and source of drizzle element in your code.

@Sukairo-02 thank you for looking into it, my issue looks to be the same and gives the same error (also using select + from) and quite easy to replicate, see: https://github.com/drizzle-team/drizzle-orm/issues/4069#issuecomment-2692199268

Minimal tsconfig:

{
    "compilerOptions": {
        "target": "ESNext",
        "module": "ESNext",
        "moduleResolution": "Bundler",
        "erasableSyntaxOnly": true,
        "strict": true,
        "skipLibCheck": true,
        "types": ["node"],
        "jsx": "react-jsx",
        "incremental": true,
        "isolatedModules": true,
        "jsxImportSource": "react",
        "strictNullChecks": true
    },
    "exclude": ["node_modules"]
}

Not sure what you mean with "source of drizzle", but using postgress: import { drizzle } from 'drizzle-orm/postgres-js' dialect: 'postgresql'

[zstiggz]

Same issue, here's a snippet:

import {
  ColumnDataType,
  SQL,
  eq,
  getTableName,
  inArray,
  sql,
} from 'drizzle-orm'
import type { PgColumn, PgTable, PgTableWithColumns } from 'drizzle-orm/pg-core'

type PickTsTableNamesFromSchema<T extends Record<string, unknown>> = {
  [K in keyof T]: T[K] extends PgTable ? K : never
}[keyof T]

export type BaseType = PgTableWithColumns<{
  name: string
  schema: string | undefined
  dialect: string
  // TODO: FIX ANY
  // would prefer Record<string, AnyPgColumn> - but that doesn't work
  columns: Record<string, any> & {
    id: PgColumn<{
      name: 'id'
      tableName: string
      dataType: ColumnDataType
      columnType: 'PgUUID' | 'PgSerial'
      data: number | string
      driverParam: number | string
      notNull: boolean
      hasDefault: boolean
      isPrimaryKey: boolean
      isAutoincrement: boolean
      hasRuntimeDefault: boolean
      enumValues: undefined
    }>
  }
}>

export function Table<C, T extends BaseType, S extends T['$inferSelect']>(
  table: T,
) {
  type I = T['$inferInsert']
  class BaseTable {

    static findByIds<R extends S>(ids: string[]): Promise<R[]> {
      return this.ctx('reader')
        .select()
        // error on this line:
       // Argument of type 'T' is not assignable to parameter of type 'TableLikeHasEmptySelection<T> extends true ? DrizzleTypeError<"Cannot reference a data-modifying statement subquery if it doesn't contain a `returning` clause"> : T'.
        .from(this.table) 
        .where(inArray(this.table.id, ids))
        .then((rows) => rows as R[])
    }
}

[geoapis-ti]

I'm trying to switch from Sequelize to drizzle and I'm also having the same issue. I rely on generics a lot to reduce a lot of boilerplate. If I'm not able to do that, then I think this migration will have to wait a while.

[wottpal]

@Sukairo-02 are you still on it? Do you need more reproductions?

[AdamHerrmann]

@Sukairo-02 not to pile on but I just ran into this issue as well. I modified your code from https://github.com/drizzle-team/drizzle-orm/issues/4069#issuecomment-2821015117 to reproduce the issue I'm seeing. This compiles in 0.38.4 but not in 0.39.0.

import {eq} from 'drizzle-orm';
import {integer, pgTable, text} from 'drizzle-orm/pg-core';
import {drizzle} from 'drizzle-orm/postgres-js';

export const users = pgTable('users', {
  id: integer('id').primaryKey().notNull(),
  name: text('name').notNull(),
});

export const systems = pgTable('systems', {
  id: integer('id').primaryKey().notNull(),
  name: text('name').notNull(),
});

type NamedTables = typeof users | typeof systems;

const db = drizzle.mock();

export async function findByName<TTable extends NamedTables>(
  table: TTable,
  name: string,
) {
  return db.select().from(table).where(eq(table.name, name));
}

Argument of type 'TTable' is not assignable to parameter of type 'TableLikeHasEmptySelection<TTable> extends true ? DrizzleTypeError<"Cannot reference a data-modifying statement subquery if it doesn't contain a `returning` clause"> : TTable'.
  Type 'NamedTables' is not assignable to type 'TableLikeHasEmptySelection<TTable> extends true ? DrizzleTypeError<"Cannot reference a data-modifying statement subquery if it doesn't contain a `returning` clause"> : TTable'.
    Type 'PgTableWithColumns<{ name: "users"; schema: undefined; columns: { id: PgColumn<{ name: "id"; tableName: "users"; dataType: "number"; columnType: "PgInteger"; data: number; driverParam: string | number; notNull: true; hasDefault: false; isPrimaryKey: true; isAutoincrement: false; hasRuntimeDefault: false; enumValues: undefined; baseColumn: never; identity: undefined; generated: undefined; }, {}, {}>; name: PgColumn<{ name: "name"; tableName: "users"; dataType: "string"; columnType: "PgText"; data: string; driverParam: string; notNull: true; hasDefault: false; isPrimaryKey: false; isAutoincrement: false; hasRuntimeDefault: false; enumValues: [string, ...string[]]; baseColumn: never; identity: undefined; generated: undefined; }, {}, {}>; }; dialect: "pg"; }>' is not assignable to type 'TableLikeHasEmptySelection<TTable> extends true ? DrizzleTypeError<"Cannot reference a data-modifying statement subquery if it doesn't contain a `returning` clause"> : TTable'.ts(2345)

[AndriiSherman]

Hey everyone!

I've created this message to send in a batch to all opened issues we have, just because there are a lot of them and I want to update all of you with our current work, why issues are not responded to, and the amount of work that has been done by our team over ~8 months.

I saw a lot of issues with suggestions on how to fix something while we were not responding – so thanks everyone. Also, thanks to everyone patiently waiting for a response from us and continuing to use Drizzle!

We currently have 4 major branches with a lot of work done. Each branch was handled by different devs and teams to make sure we could make all the changes in parallel.

---

First branch is drizzle-kit rewrite

All of the work can be found on the alternation-engine branch. Here is a PR with the work done: https://github.com/drizzle-team/drizzle-orm/pull/4439

As you can see, it has 167k added lines of code and 67k removed, which means we've completely rewritten the drizzle-kit alternation engine, the way we handle diffs for each dialect, together with expanding our test suite from 600 tests to ~9k test units for all different types of actions you can do with kit. More importantly, we changed the migration folder structure and made commutative migrations, so you won't face complex conflicts on migrations when working in a team.

What's left here:

• We are finishing handling defaults for Postgres, the last being geometry (yes, we fixed the srid issue here as well).
• We are finishing commutative migrations for all dialects.
• We are finishing up the command, so the migration flow will be as simple as drizzle-kit up for you.

Where it brings us:

• We are getting drizzle-kit into a new good shape where we can call it [email protected]!

Timeline:

• We need ~2 weeks to finish all of the above and send this branch to beta for testing.

Second big branch is a complex one with several HUGE updates

• Bringing Relational Queries v2 finally live. We've done a lot of work here to actually make it faster than RQBv1 and much better from a DX point of view. But in implementing it, we had to make another big rewrite, so we completely rewrote the drizzle-orm type system, which made it much simpler and improved type performance by ~21.4x:

(types instantiations for 3300 lines production drizzle schema + 990 lines relations)

TS v5.8.3: 728.8k -> 34.1k
TS v5.9.2: 553.7k -> 25.4k

You can read more about it here.

What's left here:

• We have 1 issue with TS that is already in progress of being fixed. The issue and Post about fixing.

Where it brings us:

• We are getting drizzle-orm into a new good shape where we can call it [email protected]!

Breaking changes:

• We will have them, but we will have open channels for everyone building on top of drizzle types, so we can guide you through all the changes.

Third branch is adding support for CockroachDB and MSSQL dialects

Support for them is already in the alternation-engine branch and will be available together with the drizzle-kit rewrite.

Summary

All of the work we are doing is crucial and should be done sooner rather than later. We've received a lot of feedback and worked really hard to find the best strategies and decisions for API, DX, architecture, etc., so we can confidently mark it as v1 and be sure we can improve it and remain flexible for all the features you are asking for, while becoming even better for everyone building on top of the drizzle API as well.

We didn't want to stay with some legacy decisions and solutions we had, and instead wanted to shape Drizzle in a way that will be best looking ahead to 2025–2026 trends (v1 will get proper effect support, etc.).

We believe that all of the effort we've put in will boost Drizzle and benefit everyone using it.

Thanks everyone, as we said, we are here to stay for a long time to build a great tool together!

Timelines

We are hoping to get v1 for drizzle in beta this fall and same timeline for latest. Right after that we can go through all of the issues and PRs and resond everyone. v1 for drizzle should close ~70% of all the bug tickets we have, so on beta release we will start marking them as closed!