[FEATURE]: How much support for cockroachdb?

[mthmcalixto]

trafficstars

Describe what you want

I like to use cockroachdb because it's serverless but I found out it's not supported yet, how much will it be?

[Enalmada]

I too am curious given I thought it was a drop in replacement for Postgres.

[a-eid]

it seem to be working for me personally, not sure what limitations it has however.

[dankochetov]

If it's working for you in Postgres mode, you can use it for sure. It's not "officially" supported yet, though, which means that if you start facing any CockroachDB-specific issues, we probably won't address them yet.

[a-eid]

@dankochetov thank you for the clarification, could you please let us know if support is in active development ?

[mbrimmer83]

I'm gonna try it out. I bet most things will work in Postgres Mode and you just have to watch out for some of the things CockroachDB doesn't support like some complex queries. I'd probably use Prisma to manage migrations until it is supported.

[roboncode]

Would be nice to support migrations.

[aydrian]

Hello. I'm a Dev Advocate at CockroachDB. I've started investigating our Drizzle support. Would love to hear your experiences. CockroachDB is postgres compatibile but doesn't support 100% of the features, like store procedures or triggers. For most things, you should be just fine using Drizzle. I did log #1267 to help it work better with CockroachDB and other distributed databases.

I'm also seeing an issue after I add a foreign key. We don't support the following:

DO $$ BEGIN
 ALTER TABLE "count_downs" ADD CONSTRAINT "count_downs_id_lists_id_fk" FOREIGN KEY ("id") REFERENCES "lists"("id") ON DELETE no action ON UPDATE no action;
EXCEPTION
 WHEN duplicate_object THEN null;
END $$;

It probably has something to do with not supporting stored procedures, but we are working on that. The above can be rewritten using an IF NOT EXISTS.

[Enalmada]

Thanks @aydrian for logging the issue. I did confirm rewriting what drizzle outputs by default seems to work:

// Drizzle default
DO $$ BEGIN
 CREATE TYPE "status" AS ENUM('ACTIVE', 'COMPLETED');
EXCEPTION
 WHEN duplicate_object THEN null;
END $$;

// Manual rewrite
CREATE TYPE IF NOT EXISTS "status" AS ENUM ('ACTIVE', 'COMPLETED');

I have a simple test app but for anyone thinking of playing around with Drizzle and CockroachDB, everything did seem to work with Drizzle after rewriting some migrations.

Issues like this with your very first interaction will probably stop most people planning to go to production with Drizzle from continuing with CockroachDB. Getting prioritization for something in CockroachDB that at least handles this DO duplicate_object would go a long way towards giving people confidence that most things should be just fine.

[aydrian]

Curious as to why you're using the DO for your example above when the IF NOT EXISTS works? I realize not every statement has this.

I'll try to see who kind of effort and priority support for duplicate_object has on my end.

[Enalmada]

@aydrian the DO is what drizzle migrate outputs by default and I wanted to put an exact before and after but I probably used confusing wording in my post.

[robinbraemer]

drizzle-kit is not supported for cockroachdb. It's a must for even considering migrating from Prisma db pull.

drizzle-kit introspect:pg
drizzle-kit: v0.20.10
drizzle-orm: v0.29.3

No config path provided, using default path
Reading config file 'website/drizzle.config.ts'
[✓] 9 tables fetched
[⣷] 0 columns fetching
[⣷] 0 enums fetching
[⣷] 0 indexes fetching
[⣷] 0 foreign keys fetching
website/node_modules/.pnpm/[email protected]/node_modules/drizzle-kit/bin.cjs:24462
              Error.captureStackTrace(err2);
                    ^

error: at or near "from": syntax error
    at website/node_modules/.pnpm/[email protected]/node_modules/drizzle-kit/bin.cjs:24462:21
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async PgPostgres.query (website/node_modules/.pnpm/[email protected]/node_modules/drizzle-kit/bin.cjs:25423:21)
    at async website/node_modules/.pnpm/[email protected]/node_modules/drizzle-kit/bin.cjs:12462:35 {
  length: 360,
  severity: 'ERROR',
  code: '42601',
  detail: 'source SQL:\n' +
    'SELECT a.attrelid::regclass::text, a.attname, is_nullable, a.attndims as array_dimensions\n' +
    "        , CASE WHEN a.atttypid = ANY ('{int,int8,int2}'::regtype[])\n" +
    '             AND EXISTS (\n' +
    '                SELECT FROM pg_attrdef ad\n' +
    '                       ^',
  hint: 'try \\h SELECT',
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'lexer.go',
  line: '404',
  routine: 'Error'
}

Node.js v18.17.1

[MikeTeddyOmondi]

Hello folks migrations didn't work for me some three weeks ago Is support for drizzle-kit migrations stable?

[DrJKL]

Getting this error with a CockroachDB database when running drizzle-kit introspect

error: could not parse "{int,int8,int2}" as type regtype[]: could not parse "int" as type oid: strconv.ParseInt: parsing "int": invalid syntax

~Any way to turn on verbose logging during introspection?~ Just added my own log statements :-)

Relevant line, I think:

        WHEN a.atttypid = ANY ('{int,int8,int2}'::regtype[]) 

When run in an ephemeral postgres instance:

select ('{int,int8,int2}'::regtype[]);

returns

{
  "regtype": "{integer,bigint,smallint}"
}

Potentially relevant: https://github.com/cockroachdb/cockroach/issues/58252

[AndriiSherman]

Hey everyone!

I've created this message to send in a batch to all opened issues we have, just because there are a lot of them and I want to update all of you with our current work, why issues are not responded to, and the amount of work that has been done by our team over ~8 months.

I saw a lot of issues with suggestions on how to fix something while we were not responding – so thanks everyone. Also, thanks to everyone patiently waiting for a response from us and continuing to use Drizzle!

We currently have 4 major branches with a lot of work done. Each branch was handled by different devs and teams to make sure we could make all the changes in parallel.

---

First branch is drizzle-kit rewrite

All of the work can be found on the alternation-engine branch. Here is a PR with the work done: https://github.com/drizzle-team/drizzle-orm/pull/4439

As you can see, it has 167k added lines of code and 67k removed, which means we've completely rewritten the drizzle-kit alternation engine, the way we handle diffs for each dialect, together with expanding our test suite from 600 tests to ~9k test units for all different types of actions you can do with kit. More importantly, we changed the migration folder structure and made commutative migrations, so you won't face complex conflicts on migrations when working in a team.

What's left here:

• We are finishing handling defaults for Postgres, the last being geometry (yes, we fixed the srid issue here as well).
• We are finishing commutative migrations for all dialects.
• We are finishing up the command, so the migration flow will be as simple as drizzle-kit up for you.

Where it brings us:

• We are getting drizzle-kit into a new good shape where we can call it [email protected]!

Timeline:

• We need ~2 weeks to finish all of the above and send this branch to beta for testing.

Second big branch is a complex one with several HUGE updates

• Bringing Relational Queries v2 finally live. We've done a lot of work here to actually make it faster than RQBv1 and much better from a DX point of view. But in implementing it, we had to make another big rewrite, so we completely rewrote the drizzle-orm type system, which made it much simpler and improved type performance by ~21.4x:

(types instantiations for 3300 lines production drizzle schema + 990 lines relations)

TS v5.8.3: 728.8k -> 34.1k
TS v5.9.2: 553.7k -> 25.4k

You can read more about it here.

What's left here:

• We have 1 issue with TS that is already in progress of being fixed. The issue and Post about fixing.

Where it brings us:

• We are getting drizzle-orm into a new good shape where we can call it [email protected]!

Breaking changes:

• We will have them, but we will have open channels for everyone building on top of drizzle types, so we can guide you through all the changes.

Third branch is adding support for CockroachDB and MSSQL dialects

Support for them is already in the alternation-engine branch and will be available together with the drizzle-kit rewrite.

Summary

All of the work we are doing is crucial and should be done sooner rather than later. We've received a lot of feedback and worked really hard to find the best strategies and decisions for API, DX, architecture, etc., so we can confidently mark it as v1 and be sure we can improve it and remain flexible for all the features you are asking for, while becoming even better for everyone building on top of the drizzle API as well.

We didn't want to stay with some legacy decisions and solutions we had, and instead wanted to shape Drizzle in a way that will be best looking ahead to 2025–2026 trends (v1 will get proper effect support, etc.).

We believe that all of the effort we've put in will boost Drizzle and benefit everyone using it.

Thanks everyone, as we said, we are here to stay for a long time to build a great tool together!

Timelines

We are hoping to get v1 for drizzle in beta this fall and same timeline for latest. Right after that we can go through all of the issues and PRs and resond everyone. v1 for drizzle should close ~70% of all the bug tickets we have, so on beta release we will start marking them as closed!

[oRafael07]

up