prisma-engines feat(psl-core): implement `parse_configuration_multi_file` to correct `prisma/prisma`'s `getConfig` implementation

This PR closes https://github.com/prisma/team-orm/issues/1103.

This PR changes the implementation of prisma_fmt::get_config, to actually support multiple schema files without validating them entirely. This fixes a non-retrocompatible regression introduced in https://github.com/prisma/prisma-engines/pull/4787 and discovered in https://github.com/prisma/prisma/pull/23824 (specifically, in DbPull.test.ts).

/integration

Apr 11 '24 13:04 jkomyno

WASM Query Engine file Size

Engine	This PR	Base branch	Diff
Postgres	2.141MiB	2.141MiB	0.000B
Postgres (gzip)	842.650KiB	842.649KiB	1.000B
Mysql	2.110MiB	2.110MiB	0.000B
Mysql (gzip)	829.014KiB	829.013KiB	1.000B
Sqlite	2.003MiB	2.003MiB	0.000B
Sqlite (gzip)	789.713KiB	789.712KiB	1.000B

Apr 11 '24 13:04 github-actions[bot]

CodSpeed Performance Report

Merging #4825 will not alter performance

_{Comparing feat/get-config-multi-file (48f2d45) with main (133a47f)}

Summary

✅ 11 untouched benchmarks

Apr 11 '24 13:04 codspeed-hq[bot]

Using prisma-fmt-wasm, consider the following:

./prisma/schema.prisma is a self-standing Prisma schema that is valid on its own

datasource db {
  provider = "postgresql"
  url = env("DBURL")
}

model A {
  id String @id
  b_id String @unique
  b B @relation(fields: [b_id], references: [id])
}

model B {
  id String @id
  a  A?
}

schema-1.prisma is only valid when paired with some other Prisma
```
model B {
  id String @id
  a  A?
}
```

schema-2.prisma is only valid when paired with some other Prisma

datasource db {
  provider = "postgresql"
  url = env("DBURL")
}

model A {
  id String @id
  b_id String @unique
  b B @relation(fields: [b_id], references: [id])
}

index.cjs shows how the get_config function works:

const fs = require('node:fs/promises')
const path = require('node:path')
const { get_config } = require('./prisma_schema_build')

async function main() {
  const schema = await fs.readFile(path.join(__dirname, 'prisma/schema.prisma'), 'utf-8')
  const schema1 = await fs.readFile(path.join(__dirname, 'prisma/schema-1.prisma'), 'utf-8')
  const schema2 = await fs.readFile(path.join(__dirname, 'prisma/schema-2.prisma'), 'utf-8')


const config1 = await get_config(JSON.stringify({ prismaSchema: schema1, ignoreEnvVarErrors: true }))
console.log('[get_config] schema1.prisma config:', JSON.parse(config1))

const config2 = await get_config(JSON.stringify({ prismaSchema: schema2, ignoreEnvVarErrors: true }))
console.log('[get_config] schema2.prisma config:', JSON.parse(config2))

const config = await get_config(JSON.stringify({ prismaSchema: [['schema-1.prisma', schema1], ['schema-2.prisma', schema2]], ignoreEnvVarErrors: true }))
console.log('[get_config] schema1.prisma + schema.2.prisma config:', JSON.parse(config))
}

main()

Before this PR, the output was:

${pwd}/prisma-engines/target/prisma-schema-wasm/src/prisma_schema_build.js:418
    const ret = new Error(getStringFromWasm0(arg0, arg1));
                ^

Error: {"message":"error: Type \"A\" is neither a built-in type, nor refers to another model, custom type, or enum.\n  -->  schema.prisma:3\n   | \n 2 |   id String @id\n 3 |   a  A?\n   | \n\nValidation Error Count: 1","error_code":"P1012"}
    
Node.js v20.9.0

With this PR, the output is:

[get_config] schema1.prisma config: { generators: [], datasources: [], warnings: [] }
[get_config] schema2.prisma config: {
  generators: [],
  datasources: [
    {
      name: 'db',
      provider: 'postgresql',
      activeProvider: 'postgresql',
      url: [Object],
      schemas: []
    }
  ],
  warnings: []
}
[get_config] schema1.prisma + schema.2.prisma config: {
  generators: [],
  datasources: [
    {
      name: 'db',
      provider: 'postgresql',
      activeProvider: 'postgresql',
      url: [Object],
      schemas: []
    }
  ],
  warnings: []
}

Apr 11 '24 13:04 jkomyno

I confirm that applying these changes to prisma/prisma allows us to uncomment these failing tests.

Apr 11 '24 14:04 jkomyno

✅ WASM query-engine performance won't change substantially (1.014x)

Full benchmark report

DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.20.2 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p999
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - ~50K)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     370 ms/iter       (367 ms … 377 ms)    376 ms    377 ms    377 ms
Web Assembly: Latest       455 ms/iter       (453 ms … 458 ms)    457 ms    458 ms    458 ms
Web Assembly: Current      457 ms/iter       (455 ms … 462 ms)    461 ms    462 ms    462 ms
Node API: Current          202 ms/iter       (197 ms … 213 ms)    210 ms    213 ms    213 ms

summary for movies.findMany() (all - ~50K)
  Web Assembly: Current
   2.26x slower than Node API: Current
   1.24x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  14'724 µs/iter (14'481 µs … 16'626 µs) 14'651 µs 16'626 µs 16'626 µs
Web Assembly: Latest    18'184 µs/iter (17'831 µs … 19'936 µs) 18'229 µs 19'936 µs 19'936 µs
Web Assembly: Current   18'439 µs/iter (18'252 µs … 19'368 µs) 18'462 µs 19'368 µs 19'368 µs
Node API: Current        8'084 µs/iter   (7'895 µs … 8'640 µs)  8'162 µs  8'640 µs  8'640 µs

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   2.28x slower than Node API: Current
   1.25x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   2'340 µs/iter   (2'202 µs … 3'848 µs)  2'334 µs  3'586 µs  3'848 µs
Web Assembly: Latest     2'908 µs/iter   (2'758 µs … 4'391 µs)  2'900 µs  3'815 µs  4'391 µs
Web Assembly: Current    2'891 µs/iter   (2'786 µs … 3'617 µs)  2'875 µs  3'532 µs  3'617 µs
Node API: Current        1'411 µs/iter   (1'327 µs … 1'713 µs)  1'425 µs  1'674 µs  1'713 µs

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   2.05x slower than Node API: Current
   1.24x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     574 ms/iter       (567 ms … 592 ms)    580 ms    592 ms    592 ms
Web Assembly: Latest       775 ms/iter       (772 ms … 783 ms)    783 ms    783 ms    783 ms
Web Assembly: Current      796 ms/iter       (789 ms … 812 ms)    800 ms    812 ms    812 ms
Node API: Current          478 ms/iter       (454 ms … 502 ms)    485 ms    502 ms    502 ms

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.67x slower than Node API: Current
   1.39x slower than Web Assembly: Baseline
   1.03x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  78'858 µs/iter (78'623 µs … 79'232 µs) 79'154 µs 79'232 µs 79'232 µs
Web Assembly: Latest       108 ms/iter       (108 ms … 109 ms)    109 ms    109 ms    109 ms
Web Assembly: Current      112 ms/iter       (112 ms … 113 ms)    113 ms    113 ms    113 ms
Node API: Current       62'496 µs/iter (60'938 µs … 69'956 µs) 62'083 µs 69'956 µs 69'956 µs

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.8x slower than Node API: Current
   1.42x slower than Web Assembly: Baseline
   1.04x slower than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'019 ms/iter   (1'012 ms … 1'042 ms)  1'021 ms  1'042 ms  1'042 ms
Web Assembly: Latest     1'293 ms/iter   (1'283 ms … 1'309 ms)  1'299 ms  1'309 ms  1'309 ms
Web Assembly: Current    1'315 ms/iter   (1'307 ms … 1'339 ms)  1'333 ms  1'339 ms  1'339 ms
Node API: Current          872 ms/iter       (852 ms … 905 ms)    887 ms    905 ms    905 ms

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.51x slower than Node API: Current
   1.29x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     143 ms/iter       (143 ms … 144 ms)    144 ms    144 ms    144 ms
Web Assembly: Latest       182 ms/iter       (179 ms … 184 ms)    184 ms    184 ms    184 ms
Web Assembly: Current      183 ms/iter       (182 ms … 184 ms)    184 ms    184 ms    184 ms
Node API: Current          107 ms/iter       (106 ms … 107 ms)    107 ms    107 ms    107 ms

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.72x slower than Node API: Current
   1.28x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'049 µs/iter     (978 µs … 1'869 µs)  1'041 µs  1'713 µs  1'869 µs
Web Assembly: Latest     1'390 µs/iter   (1'301 µs … 2'142 µs)  1'374 µs  2'098 µs  2'142 µs
Web Assembly: Current    1'421 µs/iter   (1'309 µs … 2'393 µs)  1'382 µs  2'322 µs  2'393 µs
Node API: Current          766 µs/iter     (715 µs … 1'110 µs)    799 µs    901 µs  1'110 µs

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.86x slower than Node API: Current
   1.36x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'027 µs/iter     (988 µs … 1'532 µs)  1'035 µs  1'352 µs  1'532 µs
Web Assembly: Latest     1'362 µs/iter   (1'304 µs … 1'852 µs)  1'371 µs  1'744 µs  1'852 µs
Web Assembly: Current    1'380 µs/iter   (1'319 µs … 1'941 µs)  1'387 µs  1'762 µs  1'941 µs
Node API: Current          766 µs/iter     (720 µs … 1'323 µs)    783 µs    959 µs  1'323 µs

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.8x slower than Node API: Current
   1.34x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

After changes in 48f2d456259290d8465828a5359851207bc6f9ed

Apr 11 '24 14:04 github-actions[bot]