webpack compile fails at middleware.ts with keystone in next@13 [edge readiness]

[Thinkscape]

This is might be part of general edge readiness effort; unsure if it belongs in bugs or feature requests. Feel free to have a read and move over 🤷

Repro steps

1. [email protected] and @keystone-6/[email protected]
2. Have src/middleware.ts which imports any keystone assets (i.e. the schema, or keystone config)
3. Try to retrieve keystone context i.e. for access control.

Expected

1. Compilation succeeds.

Actual

In next@13 middleware is always run in the edge runtime, hence can't use any node modules or features. Keystone seems to non-optionally require @aws-sdk which is not edge-friendly, which breaks things.

There's a discussion around this type of failure here: https://github.com/vercel/next.js/issues/28774#issuecomment-1526897155

There are other threads around being able to opt-out of edge for middleware, however I don't see that happening any time soon in Next.

dev:next: wait  - compiling /src/middleware (client and server)...
dev:next: error - node:fs
dev:next: Module build failed: UnhandledSchemeError: Reading from "node:fs" is not handled by plugins (Unhandled scheme).
dev:next: Webpack supports "data:" and "file:" URIs by default.
dev:next: You may need an additional plugin to handle "node:" URIs.
dev:next: Import trace for requested module:
dev:next: node:fs
dev:next: ./node_modules/.pnpm/[email protected]/node_modules/strtok3/lib/FsPromise.js
dev:next: ./node_modules/.pnpm/[email protected]/node_modules/strtok3/lib/index.js
dev:next: ./node_modules/.pnpm/[email protected]/node_modules/file-type/index.js
dev:next: ./node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected]_@opentelemetry+ap_eexecvfunafgo67t2ljvtiogxm/node_modules/@keystone-6/core/dist/createSystem-a93ebf95.esm.js
dev:next: ./node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected]_@opentelemetry+ap_eexecvfunafgo67t2ljvtiogxm/node_modules/@keystone-6/core/context/dist/keystone-6-core-context.esm.js
dev:next: ./src/keystone/context.ts
dev:next: ./src/server/auth.ts

Workaround ?

The only way is to avoid any cross imports between middleware.ts and Keystone, which makes it impossible to use keystone queries for access control, which is a major issue for me 😞

[mayank132]

facing the same issue not getting any answer on this,what should I do to get the keystone context session

[dcousens]

@Thinkscape is @aws-sdk the only module that prevents this for you?

[Thinkscape]

@Thinkscape is @aws-sdk the only module that prevents this for you?

Unfortunately not... tried excluding it from the build, but there's others. It is a bit hopeless because of hard-dep on Prisma. I know that team is working on a big refactor to make it happen, but it might be some time before Prisma is usable outside of Node.js.

[dcousens]

@Thinkscape what is your expectation in any event, short-lived POSTGRES connections? Or something like https://neon.tech/blog/how-to-use-postgres-at-the-edge ?

[Thinkscape]

Yep. I've moved on from KS/Prisma to a Next/Edge/Neon/Drizzle/Upstash stack. Majority of requests are one-off, non-transactional queries anyways, so all the http drivers (drizzle-orm/neon-http being one of them) work perfectly fine, scale well, cost is predictable and reasonable. Looking back at KS apps, same thing applies - it's just a flurry of selects that would be fine over https. For some complex operations, when a session is required, there's always the tcp connection fallback, but I can use the same Drizzle schema (just switch driver).

Because of Vercel's Edge push, Next.js is quite competent on its own in this regard, and the above complement it well. Occasionally there's a library or two that do not work well on edge runtime (surprising one for me was lodash, or parts thereof), but alternatives are aplenty and sometimes I patch out the conflicting bits (like accessing window. properties or a cheeky eval() where it shouldn't).

That reminds me, maybe I should grab a cold one with yourself and Jed to discuss. Not sure how much you've spent researching this "edge" trend so far at TM. Theo has been pushing it, partial prerenders will make things ever better when compounded with the above (fast cdn fragments + zero cold boots of edge). 🤷‍♂️