graphql-request icon indicating copy to clipboard operation
graphql-request copied to clipboard
verified publisher icon jasonkuhrt

Migrate to monorepo structure

Open jasonkuhrt opened this issue 2 months ago • 1 comments

Migrate to Monorepo Structure

Overview

Transform Graffle from a single package into a monorepo with 17 focused packages. This will:

  • ✅ Enable independent versioning and releases
  • ✅ Separate generic GraphQL utilities (@graffle/grafaid) from Graffle-specific code
  • ✅ Allow extensions to be used independently
  • ✅ Improve tree-shaking and bundle sizes
  • ✅ Enable better code organization and maintainability
  • ✅ Claim the @graffle npm scope

Package Structure (17 packages)

Core Packages (5)

1. @graffle/core

Graffle-specific shared infrastructure

Contains:

  • Context system (src/context/)
  • Extension base class (Extension)
  • GlobalRegistry
  • TypeFunction
  • ContextFragments
  • RequestResult types
  • Docpar (document parser)
  • SchemaDrivenDataMap
  • Shared Graffle data structures

Used by: client, generator, all extensions

2. @graffle/grafaid

Generic GraphQL utilities (not Graffle-specific)

Contains:

  • Schema types & definitions (from src/types/Schema/)
  • GraphQL document utilities
  • Schema introspection & analysis
  • HTTP helpers
  • Execution helpers
  • Tree-shakeable, reusable GraphQL tools

Design Goal: Grafaid should be usable standalone for any GraphQL tooling, not just Graffle.

Used by: client, generator

3. @graffle/client

GraphQL Client

Contains:

  • Client implementation (src/client/)
  • Request pipeline (src/requestPipeline/)
  • Static helpers (src/static/gql)
  • Client methods
  • Export path: graffle/utilities-for-generated (re-exports from core + client for generated code)

Depends on: @graffle/core, @graffle/grafaid

4. @graffle/generator

Code Generator & CLI

Contains:

  • Generator (src/generator/)
  • CLI (src/cli/)
  • Code generation logic

Depends on: @graffle/grafaid, @graffle/core

5. graffle

Main Convenience Package

The "batteries included" package.

  • Re-exports @graffle/client and common extensions
  • Default entry point for users: npm i graffle

Extension Packages (8)

Each extension is independently publishable and versioned:

  • @graffle/extension-document-builder
  • @graffle/extension-introspection
  • @graffle/extension-throws
  • @graffle/extension-opentelemetry
  • @graffle/extension-schema-errors
  • @graffle/extension-transport-http
  • @graffle/extension-transport-memory
  • @graffle/extension-upload

All depend on: @graffle/core

Preset Packages (3)

Pre-configured extension bundles:

  • @graffle/preset-bare
  • @graffle/preset-basic
  • @graffle/preset-minimal

Dependency Graph

@wollybeard/kit (external)
          ↓
    @graffle/core
          ↓
    ┌─────┴─────────────┐
    ↓                   ↓
@graffle/grafaid   Extensions (8)
    ↓                   ↓
@graffle/client    Presets (3)
@graffle/generator      ↓
    ↓                   ↓
  graffle (meta package)

Key Design Decisions

1. Grafaid = Generic, Core = Graffle-specific

  • @graffle/grafaid: Generic GraphQL utilities, tree-shakeable, usable standalone
  • @graffle/core: Graffle-specific infrastructure (extensions, context, etc.)

2. Replace Internal Kit with @wollybeard/kit

  • Remove src/lib/anyware, src/lib/tex, src/lib/config-manager
  • Use @wollybeard/kit for these utilities

3. Keep utilities-for-generated as Export Path

  • Not a separate package
  • Export path in @graffle/client: graffle/utilities-for-generated
  • Generated code imports: import * as $$Utilities from "graffle/utilities-for-generated"

4. Independent Versioning

  • Each package can be released independently
  • Use Changesets for version management

5. Schema Types in Grafaid

  • Move src/types/Schema/ into @graffle/grafaid
  • Rely on tree-shaking to keep client bundle small

Tooling Stack

Build & Task Running

  • pnpm workspaces - Package management
  • Turborepo - Build orchestration & caching (temporary)
  • TypeScript project references - Fast, incremental builds
  • Vite/Rolldown - Bundling

Future Migration

  • Migrate to Vite+ when available (early 2026)
  • Vite+ includes built-in monorepo task runner with intelligent caching

Versioning & Publishing

  • Changesets - Independent versioning & changelog generation
  • Each package publishes to @graffle/* scope (except main graffle package)

Directory Structure

graffle/                           # Monorepo root
├── packages/
│   ├── core/                      # @graffle/core
│   ├── grafaid/                   # @graffle/grafaid
│   ├── client/                    # @graffle/client
│   ├── generator/                 # @graffle/generator
│   ├── graffle/                   # graffle (meta package)
│   │
│   ├── extensions/
│   │   ├── document-builder/      # @graffle/extension-document-builder
│   │   ├── introspection/         # @graffle/extension-introspection
│   │   ├── throws/                # @graffle/extension-throws
│   │   ├── opentelemetry/         # @graffle/extension-opentelemetry
│   │   ├── schema-errors/         # @graffle/extension-schema-errors
│   │   ├── transport-http/        # @graffle/extension-transport-http
│   │   ├── transport-memory/      # @graffle/extension-transport-memory
│   │   └── upload/                # @graffle/extension-upload
│   │
│   └── presets/
│       ├── bare/                  # @graffle/preset-bare
│       ├── basic/                 # @graffle/preset-basic
│       └── minimal/               # @graffle/preset-minimal
│
├── examples/                      # Example projects (workspace, not published)
├── website/                       # Documentation site (workspace, not published)
├── tests/                         # Integration/e2e tests
├── scripts/                       # Build/dev scripts
├── turbo.json                     # Turborepo config
├── pnpm-workspace.yaml            # pnpm workspace config
└── package.json                   # Root package.json

Migration Strategy

Phase 1: Setup Infrastructure

  • Create monorepo structure
  • Configure pnpm workspaces
  • Set up Turborepo
  • Configure TypeScript project references
  • Set up Changesets

Phase 2: Split Core Packages

  • Extract @graffle/core
  • Extract @graffle/grafaid (includes Schema types)
  • Split @graffle/client
  • Split @graffle/generator
  • Update internal imports

Phase 3: Extract Extensions

  • Move each extension to packages/extensions/*
  • Update extension imports and dependencies
  • Test extensions independently

Phase 4: Extract Presets

  • Move presets to packages/presets/*
  • Update preset dependencies

Phase 5: Create Meta Package

  • Create packages/graffle/ as main entry point
  • Re-export client + common extensions
  • Update documentation

Phase 6: Replace Kit Dependencies

  • Replace src/lib/anyware → @wollybeard/kit
  • Replace src/lib/tex → @wollybeard/kit
  • Replace src/lib/config-manager → @wollybeard/kit
  • Remove internal lib utilities

Phase 7: Testing & Publishing

  • Run full test suite across all packages
  • Update CI/CD for monorepo
  • Publish initial versions
  • Update documentation and migration guides

References

  • Related: #1124 (Extract extensions to own packages)
  • npm scope @graffle has been claimed

Open Questions

  • [ ] Should examples/ and website/ be in monorepo or separate repos? Decision: Keep in monorepo
  • [ ] Versioning strategy for extensions - lock-step or independent? Decision: Independent
  • [ ] Breaking change migration path for users

jasonkuhrt avatar Oct 25 '25 01:10 jasonkuhrt

Detailed Monorepo Migration Plan - Updated 2025-10-28

Strategy: Big Bang Migration with Independent Versioning

Approach: Complete monorepo transformation releasing 14 packages simultaneously with independent versioning.

Key Decisions:

  • @graffle/grafaid → @graffle/graphql (clearer name for generic GraphQL utilities)
  • New @graffle/document package (docpar + document utilities extracted from core)
  • Presets bundled in graffle meta package (not separate packages)
  • 14 total packages (5 core + 8 extensions + 1 meta)
  • Build tool: Turborepo (temporary, migrate to Vite+ in 2026)
  • Versioning: Independent from day one using Changesets
  • User migration: Manual with comprehensive documentation

Package Structure (14 packages)

Core Packages (5)

  1. @graffle/graphql - Generic GraphQL utilities (was grafaid)
  2. @graffle/document - Document parsing (docpar, SDDM, Select, etc.)
  3. @graffle/core - Graffle infrastructure (context, extensions)
  4. @graffle/client - GraphQL client
  5. @graffle/generator - Code generator & CLI

Extension Packages (8)

  1. @graffle/extension-document-builder
  2. @graffle/extension-introspection
  3. @graffle/extension-throws
  4. @graffle/extension-opentelemetry
  5. @graffle/extension-schema-errors
  6. @graffle/extension-transport-http
  7. @graffle/extension-transport-memory
  8. @graffle/extension-upload

Meta Package (1)

  1. graffle - Batteries included (includes presets as export paths)

Dependency Graph

@wollybeard/kit (external)
          ↓
   @graffle/graphql (GraphQL utilities, schema types)
          ↓
   @graffle/document (docpar, SDDM, Select)
          ↓
   @graffle/core (context, extensions)
          ↓
    ┌─────┴─────────────┐
    ↓                   ↓
@graffle/client    Extensions (8)
@graffle/generator      
    ↓                   ↓
  graffle (meta + presets)

Phase 1: Monorepo Infrastructure Setup

Goal: Establish monorepo tooling

1.1 Initialize Workspace Structure

packages/
├── graphql/                    # @graffle/graphql
├── document/                   # @graffle/document
├── core/                       # @graffle/core
├── client/                     # @graffle/client
├── generator/                  # @graffle/generator
├── extensions/
│   ├── document-builder/
│   ├── introspection/
│   ├── throws/
│   ├── opentelemetry/
│   ├── schema-errors/
│   ├── transport-http/
│   ├── transport-memory/
│   └── upload/
└── graffle/                    # Meta package

1.2 Configure pnpm Workspaces

  • Create pnpm-workspace.yaml
  • Move root dependencies to appropriate packages

1.3 Setup Turborepo

  • Create turbo.json with pipeline configuration
  • Configure caching for build outputs

1.4 TypeScript Project References

  • tsconfig.base.json for shared config
  • Each package: tsconfig.json + tsconfig.build.json
  • Configure composite builds

1.5 Setup Changesets

  • Install @changesets/cli
  • Configure for independent versioning
  • Add version/publish scripts

1.6 Update CI/CD

  • GitHub Actions for monorepo
  • Turbo for builds
  • Changeset automation

Dependencies: None
Validation: Empty packages build

Phase 2: Extract @graffle/graphql (Foundation)

Goal: Generic GraphQL utilities, zero internal dependencies

  • Move src/lib/grafaid/** → packages/graphql/src/
  • Move src/types/Schema/** → packages/graphql/src/schema-types/
  • Setup tree-shakeable exports
  • Dependencies: External only (@0no-co/graphql.web, @wollybeard/kit, peer graphql)

Dependencies: Phase 1
Validation: Builds and tests independently

Phase 3: Extract @graffle/document (Document Layer)

Goal: Document parsing and selection set utilities

  • Move src/docpar/** → packages/document/src/
  • Setup exports for SDDM, Select, InferResult
  • Dependencies: @graffle/graphql

Dependencies: Phase 2
Validation: Builds independently, imports from @graffle/graphql work

Phase 4: Extract @graffle/core (Infrastructure)

Goal: Graffle-specific shared infrastructure

  • Move src/context/** → packages/core/src/context/
  • Move src/types/RequestResult/**, GlobalRegistry/**, ContextFragment.ts
  • Move src/utils.ts
  • Extract Extension base class
  • Resolve circular dependencies: Extract HandleOutput type to core

Dependencies: Phase 3
Validation: Core builds, no circular dependency errors

Phase 5: Extract @graffle/client (Client)

Goal: GraphQL client implementation

  • Move src/client/**, requestPipeline/**, static/**, select/**
  • CRITICAL: Create utilities-for-generated export:
    • Re-exports from core, document, graphql
    • Maintains API for generated code
  • Dependencies: @graffle/core, @graffle/document, @graffle/graphql

Dependencies: Phase 4
Validation: Client builds, utilities-for-generated exports correctly

Phase 6: Extract @graffle/generator (Generator)

Goal: Code generator and CLI

  • Move src/generator/**, cli/**
  • Move src/lib/fsp.ts, import-first.ts, typescript-formatter.ts
  • Update generated code templates to use @graffle/* imports
  • Dependencies: @graffle/core, @graffle/document, @graffle/graphql

Dependencies: Phase 5
Validation: Generator outputs correct imports

Phase 7: Extract Extensions (8 Packages)

Migration Order:

  1. Simple: throws, opentelemetry, upload, transport-memory
  2. Transport: transport-http
  3. Complex: introspection
  4. Most complex: document-builder, schema-errors

For each extension:

  • Create package structure
  • Move source from src/extensions/{Name}/**
  • Setup exports (runtime + gentime for SchemaErrors)
  • Update dependencies and imports
  • DO NOT EDIT generated fixtures

Special handling:

  • DocumentBuilder circular dependency: Extract graffleMappedResultToRequest to @graffle/core/utilities

Dependencies: Phase 6
Validation: Each builds independently

Phase 8: Create Meta Package with Presets

Goal: Batteries-included package with bundled presets

  • Move src/exports/presets/*.ts → packages/graffle/src/presets/
  • Create re-exports for client and common extensions
  • Setup export paths matching old structure:
    • graffle → main
    • graffle/presets/basic → basic preset
    • graffle/extensions/* → extensions (backward compatible)
  • Dependencies: @graffle/client + common extensions

Dependencies: Phase 7
Validation: Meta package installs, imports work

Phase 9: Regenerate Test Fixtures

Goal: All generated code uses new structure

  • Regenerate all test schemas
  • Regenerate SchemaErrors/DocumentBuilder fixtures
  • Regenerate website schemas (Pokemon, Graffle)
  • Regenerate examples
  • Run full test suite: turbo run test

Dependencies: Phase 8
Validation: All tests pass, generated code correct

Phase 10: Documentation & Migration

Goal: User migration guide

Migration Guide

Old → New:

// Old
import { Graffle } from 'graffle'
import { SchemaErrors } from 'graffle/extensions/schema-errors'

// New (Option 1: Meta package - easiest)
import { Graffle } from 'graffle'
import { SchemaErrors } from '@graffle/extension-schema-errors'
// OR keep: import { SchemaErrors } from 'graffle/extensions/schema-errors'

// New (Option 2: Specific packages - smaller bundles)
import { Graffle } from '@graffle/client'
import { SchemaErrors } from '@graffle/extension-schema-errors'

// Presets unchanged
import { create } from 'graffle/presets/basic'

Dependencies: Phase 9
Validation: Docs complete, examples work

Phase 11: Publishing Prep

  • Verify all package.json metadata
  • Run publint on all packages
  • Local testing with pnpm link
  • Create initial changeset (major version, all 14 packages)
  • Verify @graffle npm scope access
  • Dry run publish

Dependencies: Phase 10
Validation: Publint passes, dry-run succeeds

Phase 12: CI/CD & Release

  • Update GitHub Actions workflows
  • Test CI pipeline
  • Merge monorepo PR to main
  • Review and merge version PR
  • Automated publish to npm
  • Verify all 14 packages published

Dependencies: Phase 11
Validation: Successful publish

Risk Mitigation

  1. Circular dependencies - Resolved in Phase 4 by extracting shared types to core
  2. Generated code breaking - Thorough testing in Phase 9
  3. Import path changes - Backward compatible paths in meta package + migration guide
  4. Publishing failures - Dry-run testing in Phase 11

Rollback Plan: Work in feat/monorepo branch, only merge when fully tested

Success Criteria

  • ✅ All 14 packages build successfully with Turborepo
  • ✅ All tests pass in monorepo structure
  • ✅ Generated code uses correct imports
  • ✅ Examples work with new package structure
  • ✅ All packages published to npm under @graffle scope
  • ✅ CI/CD pipeline works with changesets
  • ✅ Documentation updated and migration guide complete
  • ✅ No circular dependency errors
  • ✅ Independent versioning works with changesets

Estimated Effort

  • Phase 1: 1-2 days (infra)
  • Phase 2: 1 day (graphql)
  • Phase 3: 1 day (document)
  • Phase 4: 2 days (core + circular deps)
  • Phase 5: 1-2 days (client)
  • Phase 6: 1-2 days (generator)
  • Phase 7: 3-4 days (8 extensions)
  • Phase 8: 1 day (meta + presets)
  • Phase 9: 1-2 days (regenerate)
  • Phase 10: 1-2 days (docs)
  • Phase 11: 1 day (publish prep)
  • Phase 12: 0.5 day (release)

Total: ~14-20 days focused work

jasonkuhrt avatar Oct 29 '25 02:10 jasonkuhrt