convex-js icon indicating copy to clipboard operation
convex-js copied to clipboard

`externalPackages` doesn't work in monorepos

Open IAlphaOmegaI opened this issue 1 year ago • 2 comments

External Packages Fail in Monorepos Due to Dependency Hoisting - Major DX Issue

Description There's a significant issue when using Convex's externalPackages for node envs feature within monorepos. The current implementation doesn't account for monorepo package management, where dependencies are typically hoisted to the root. This affects ALL external packages, with Sharp being a prime example in my case.

Problem Scope This is NOT just a Sharp-specific issue - it affects any external package used with Convex in a monorepo setup. The externalPackages feature expects packages in the local node_modules, and since monorepos hoist the dependecies to the root, convex is unable to find them.

Current Behavior When using ANY external package in a monorepo:

  1. Dependencies are hoisted to root node_modules (standard monorepo behavior)
  2. Convex can't locate the package even when it's in root package.json
  3. Deployment fails due to missing dependencies
  4. Adding to root package.json is insufficient - must also add to the Convex app's package.json

Example with Sharp

monorepo/
  ├─ node_modules/ (where packages actually live)
  ├─ apps/
  │  ├─ convex-app/
  │  │  ├─ node_modules/ (where Convex expects packages)
  │  │  ├─ package.json (needs duplicate declaration)

Current Workaround (Hacky)

  1. Create manual node_modules inside the Convex app directory
  2. Physically copy package files from root node_modules
  3. Add duplicate package entries in both root and app package.json
  4. Configure as externalPackage

Suggested Solutions

  1. Short-term: Document this limitation clearly in the externalPackages docs
  2. Medium-term: Add monorepo detection and automatic package location resolution
  3. Long-term: Redesign the externalPackages feature to work with hoisted dependencies

Environment

  • Package Managers Affected: pnpm, yarn, npm, bun (all with workspaces)
  • Monorepo Tools: Turborepo, Nx, others
  • Convex Version: ^1.14.4

IAlphaOmegaI avatar Nov 21 '24 19:11 IAlphaOmegaI