[BUG]: Nullable columns do not exist in insert(table).values()

[PhilipAngelinNE]

What version of drizzle-orm are you using?

0.32.1

What version of drizzle-kit are you using?

0.23.0

Describe the Bug

const files = pgTable("files", {
  id: uuid("id").primaryKey().defaultRandom(),
  externalId: char("externalId", { length: 30 }).notNull(),
  password: char("password", { length: 30 }),
  size: bigint("size", { mode: "number" }).notNull(), // In bytes
  expires: bigint("expires", { mode: "number" }).notNull(), // UNIX timestamp
});

this.db.insert(files).values({ externalId, size, expires, password })

Here, TS complains that password does not exist in the values object. If I add .notNull() to the password column it works.

Expected behavior

In this case, password should exist in .values().

Environment & setup

OS: WSL Ubuntu. TypeScript version: 5.4.5. Node version: 20. Using Nx mono repo with a NestJS app (where the example is used).

[PhilipAngelinNE]

I found in https://github.com/drizzle-team/drizzle-orm/issues/2654#issuecomment-2247698796 that this only seems to happen with tsconfig strict mode set to false. Nx does not set it to true, so I turned them on myself and that fixed the issue.

However I have a friend who tried doing the same, and it does still not work for him, so I'll leave this issue open.

[micaww]

seems to be an issue only 0.32+. i downgraded to 0.31.2 and the nullable properties are correctly in the insert type

[lmcneel]

seems to be an issue only 0.32+. I downgraded to 0.31.2, and the nullable properties are correctly in the insert type

Unfortunately, we ran into an issue with drizzle-kit which requires the latest version. I, too, initially downgraded until we needed to run migrations.

[krunalcodes]

I have spent like an hour trying to fix this issue. I was using v0.33 i tried downgrading to v0.32 but still didn't work. Adding the strict:true to tsconfig.json solved the issue for me.

[Karo8870]

I'm not sure if this is a framework problem, but for me setting "strictNullChecks" to true instead of "strict" did the trick. I'm using NestJS with DrizzleORM v0.33.0

[Niewdanka]

Any updates?

[danielsharvey]

@Adrian333Dev @Niewdanka Do the changes I mention in https://github.com/drizzle-team/drizzle-orm/issues/2636#issuecomment-2308722249 make any difference for you?

[Karo8870]

@Adrian333Dev Could you provide more details? I can't seem reproduce the problem. The tsconfig.json and posts.schema.ts might help.

This is my tsconfig btw:

{
  "compilerOptions": {
    "module": "commonjs",
    "declaration": true,
    "removeComments": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "allowSyntheticDefaultImports": true,
    "target": "ES2021",
    "sourceMap": true,
    "outDir": "./dist",
    "baseUrl": "./",
    "incremental": true,
    "skipLibCheck": true,
    "strictNullChecks": true,
    "noImplicitAny": false,
    "strictBindCallApply": false,
    "forceConsistentCasingInFileNames": false,
    "noFallthroughCasesInSwitch": false
  }
}

[Karo8870]

I found the problem, the circular dependency between users and posts breaks drizzle types. You currently have the following tables:

export const posts = pgTable('posts', {
  postId: serial('post_id').primaryKey(),
  content: text('content'),
  published: boolean('published').default(false),
  timestamp: timestamp('timestamp').defaultNow(),
  userId: integer('user_id').references(() => users.userId),
});

export const users = pgTable('users', {
  userId: serial('user_id').primaryKey(),
  email: text('email').unique().notNull(),
  password: text('password').notNull(),
  postId: integer('post_id').references(() => posts.postId),
});

Since posts have a column referencing users and users have a column referencing posts it causes an "infinite type loop" if I can call it that, that in the end just ends up being any.

Are you sure that the postId column should be there? Doesn't seem to make sense...

[Karo8870]

He added a github link with the final code in the description of the video. That final code doesn't have any postId column on the users table, so I'll assume that he deletes it at some point in the video after realizing it is a mistake, you can delete it too and the typescript should work.

How much experience do you have with NestJS, drizzle and relational databases?

[Adrian333Dev]

Yeah, it doesn't even make sense to add that in there, I might have added my accident. I have like 1.5 years of basic experience with NestJS, but I'm just picking up drizzle. My Bad

[sarthakSh116]

and now how can I do a partial of the insert, changing tsconfig does add all the fields but I only want to insert a subset of those

[Adrian333Dev]

and now how can I do a partial of the insert, changing tsconfig does add all the fields but I only want to insert a subset of those

Make the fields in the schema optional by not adding notNull()? I mean if the field is required in the SQL table, you have to insert it right?

[sarthakSh116]

thanks, I forgot to remove to mention. I saw it later that not adding notNull makes them optional

[sarthakSh116]

strictNullChecks is now causing a lot of errors for me on ther things, mostly while accessing .env files etc. How do I handle these and the drizzle thing to insert optional columns

[OzanOcak]

isn't any column nullable by default ? I just don't add notNull().

[ykisana]

Having this issue too. I have a column that can be null. Want to add via insert if the value is present but it's giving this error.

[jeremy-returned]

This seems to be connected to #2636

Current workaround is 1 of 2 fixes:

1. Assign a variable above with the insert values let insertItem = { nullfield: 1 }; db.insert(table).values(insertItem);
2. Add "as any" after the insert db.insert(table).values({ nullfield: 1 } as any);

[snr-lab]

Any update on this issue

[mo-cherif]

Any update on this issue

having the same issue ... in your tsconfig.json put strict to true instead of false

[snr-lab]

Any update on this issue

having the same issue ... in your tsconfig.json put strict to true instead of false

This is not really an option for me. Turing tsconfig.json into strict mode create huge number of new errors. I started the project sometime back and this issue was not earlier.

[bagusok]

i try this tsconfig.json and work for nestjs and turborepo

 "compilerOptions": {
   "module": "commonjs",
   "declaration": true,
   "removeComments": true,
   "emitDecoratorMetadata": true,
   "experimentalDecorators": true,
   "allowSyntheticDefaultImports": true,
   "target": "ES2021",
   "sourceMap": true,
   "outDir": "./dist",
   "baseUrl": "./",
   "incremental": true,
   "skipLibCheck": true,
   "strictNullChecks": true,
   "noImplicitAny": true,
   "strictBindCallApply": true,
   "forceConsistentCasingInFileNames": true,
   "noFallthroughCasesInSwitch": true
 },
 "include": ["src/**/*"],
 "exclude": ["node_modules", "dist"]
}

[okep]

i try this tsconfig.json and work for nestjs and turborepo

 "compilerOptions": {
   "module": "commonjs",
   "declaration": true,
   "removeComments": true,
   "emitDecoratorMetadata": true,
   "experimentalDecorators": true,
   "allowSyntheticDefaultImports": true,
   "target": "ES2021",
   "sourceMap": true,
   "outDir": "./dist",
   "baseUrl": "./",
   "incremental": true,
   "skipLibCheck": true,
   "strictNullChecks": true,
   "noImplicitAny": true,
   "strictBindCallApply": true,
   "forceConsistentCasingInFileNames": true,
   "noFallthroughCasesInSwitch": true
 },
 "include": ["src/**/*"],
 "exclude": ["node_modules", "dist"]
}

try: "strictNullChecks": false

[tarikhagustia]

same here +1

[CyberClarence]

this is shit

[almuth]

having the same issue, also happened to update(table).set()

[JoaquinRodriguezc]

same here, any update on this issue?

[AndriiSherman]

Hey everyone!

I've created this message to send in a batch to all opened issues we have, just because there are a lot of them and I want to update all of you with our current work, why issues are not responded to, and the amount of work that has been done by our team over ~8 months.

I saw a lot of issues with suggestions on how to fix something while we were not responding – so thanks everyone. Also, thanks to everyone patiently waiting for a response from us and continuing to use Drizzle!

We currently have 4 major branches with a lot of work done. Each branch was handled by different devs and teams to make sure we could make all the changes in parallel.

---

First branch is drizzle-kit rewrite

All of the work can be found on the alternation-engine branch. Here is a PR with the work done: https://github.com/drizzle-team/drizzle-orm/pull/4439

As you can see, it has 167k added lines of code and 67k removed, which means we've completely rewritten the drizzle-kit alternation engine, the way we handle diffs for each dialect, together with expanding our test suite from 600 tests to ~9k test units for all different types of actions you can do with kit. More importantly, we changed the migration folder structure and made commutative migrations, so you won't face complex conflicts on migrations when working in a team.

What's left here:

• We are finishing handling defaults for Postgres, the last being geometry (yes, we fixed the srid issue here as well).
• We are finishing commutative migrations for all dialects.
• We are finishing up the command, so the migration flow will be as simple as drizzle-kit up for you.

Where it brings us:

• We are getting drizzle-kit into a new good shape where we can call it [email protected]!

Timeline:

• We need ~2 weeks to finish all of the above and send this branch to beta for testing.

Second big branch is a complex one with several HUGE updates

• Bringing Relational Queries v2 finally live. We've done a lot of work here to actually make it faster than RQBv1 and much better from a DX point of view. But in implementing it, we had to make another big rewrite, so we completely rewrote the drizzle-orm type system, which made it much simpler and improved type performance by ~21.4x:

(types instantiations for 3300 lines production drizzle schema + 990 lines relations)

TS v5.8.3: 728.8k -> 34.1k
TS v5.9.2: 553.7k -> 25.4k

You can read more about it here.

What's left here:

• We have 1 issue with TS that is already in progress of being fixed. The issue and Post about fixing.

Where it brings us:

• We are getting drizzle-orm into a new good shape where we can call it [email protected]!

Breaking changes:

• We will have them, but we will have open channels for everyone building on top of drizzle types, so we can guide you through all the changes.

Third branch is adding support for CockroachDB and MSSQL dialects

Support for them is already in the alternation-engine branch and will be available together with the drizzle-kit rewrite.

Summary

All of the work we are doing is crucial and should be done sooner rather than later. We've received a lot of feedback and worked really hard to find the best strategies and decisions for API, DX, architecture, etc., so we can confidently mark it as v1 and be sure we can improve it and remain flexible for all the features you are asking for, while becoming even better for everyone building on top of the drizzle API as well.

We didn't want to stay with some legacy decisions and solutions we had, and instead wanted to shape Drizzle in a way that will be best looking ahead to 2025–2026 trends (v1 will get proper effect support, etc.).

We believe that all of the effort we've put in will boost Drizzle and benefit everyone using it.

Thanks everyone, as we said, we are here to stay for a long time to build a great tool together!

Timelines

We are hoping to get v1 for drizzle in beta this fall and same timeline for latest. Right after that we can go through all of the issues and PRs and resond everyone. v1 for drizzle should close ~70% of all the bug tickets we have, so on beta release we will start marking them as closed!