node-sdk icon indicating copy to clipboard operation
node-sdk copied to clipboard

Upgrade to Zod 4

Open Murderlon opened this issue 6 months ago • 9 comments

Customers are having compatibility issues.

Murderlon avatar Oct 29 '25 08:10 Murderlon

I think this will also introduce compatibility issues for customers who use zod3.

Should we have a dual mode? Exporting both v3 and v4 schemas? Like zod itself also exposes /v3 inside v4? Not sure what the best approach is here.

I would expect that for the inferred types, those are still the same whether your schemas are v3 or v4..

kvz avatar Oct 29 '25 14:10 kvz

@claude can you come up with a best practice proposal that addresses these concerns elegantly?

kvz avatar Oct 29 '25 14:10 kvz

Claude finished @kvz's task —— View job


Zod v4 Upgrade Strategy Analysis

I've analyzed the codebase and researched Zod v4 compatibility. Here's my comprehensive recommendation:

Todo List:

  • [x] Examine current Zod usage in codebase
  • [x] Check current Zod version and dependencies
  • [x] Research Zod v4 breaking changes and compatibility features
  • [x] Analyze dual export feasibility
  • [x] Propose migration strategy with pros/cons
  • [x] Provide implementation recommendations

Current State Analysis

  • Current Version: Zod v3.25.76
  • Usage Scale: ~96 files with extensive Zod integration
  • Key Usage Patterns:
    • Discriminated unions (z.discriminatedUnion) for 80+ robot instructions
    • Complex schemas with .strict() validation
    • Extensive z.infer<> for TypeScript type generation
    • Custom error handling via zodParseWithContext.ts
    • Mixed use of .parse() and .safeParse()

Zod v4 Key Changes & Compatibility

Breaking Changes in Zod v4:

  1. Stricter type inference - Some edge cases in z.infer<> behavior
  2. Enhanced discriminated union handling - Better performance but potential runtime differences
  3. Stricter .strict() behavior - May reject previously accepted schemas
  4. Updated error formats - Could affect custom error handling

Good News:

  • Zod v4 maintains API compatibility for core functionality
  • Most schemas will work without modification
  • Type inference improvements generally enhance rather than break existing code

Recommended Migration Strategy: Gradual Migration with Compatibility Layer

Phase 1: Dual Export Approach ✅

Create a compatibility layer that supports both Zod v3 and v4:

// src/zod-compat.ts
import { z as zodV4 } from 'zod'
import { z as zodV3 } from 'zod/v3' // Zod v4 ships with v3 compatibility

export const z = zodV4 // Default to v4
export const zv3 = zodV3 // Fallback for problematic schemas

Phase 2: Schema-by-Schema Migration

  1. Update package.json to Zod v4
  2. Test robot schemas - Start with simpler ones
  3. Update custom error handling in zodParseWithContext.ts
  4. Validate discriminated unions work correctly
  5. Run comprehensive tests

Phase 3: Customer Impact Mitigation

For SDK consumers:

  • Inferred types remain stable - customers' TypeScript won't break
  • Runtime validation should be identical for valid data
  • Error messages may improve (generally beneficial)

Implementation Timeline

// Week 1: Setup compatibility layer
export { z } from 'zod' // v4
export { z as zv3 } from 'zod/v3' // v3 fallback

// Week 2-3: Test & migrate core schemas
import { z, zv3 } from './zod-compat'
// Use zv3 only for problematic schemas

// Week 4: Remove v3 fallbacks
export { z } from 'zod' // Clean v4-only export

Risk Assessment

Low Risk ✅:

  • Type inference improvements
  • Better error messages
  • Performance enhancements

Medium Risk ⚠️:

  • Custom error handling changes
  • Discriminated union edge cases

Mitigation:

  • Comprehensive test coverage
  • Gradual rollout
  • Monitor customer feedback

Recommendation

Proceed with Zod v4 upgrade using the gradual migration approach. The dual export strategy minimizes risk while allowing you to leverage v4's improvements. Customer impact should be minimal since inferred types remain stable and runtime validation improves.

Key advantages:

  • ✅ Better performance for large discriminated unions
  • ✅ Enhanced type safety
  • ✅ Improved error messages
  • ✅ Future-proofing the codebase

The /v3 compatibility export in Zod v4 provides an excellent safety net for the transition period.


claude[bot] avatar Oct 30 '25 07:10 claude[bot]

I think claude advice was too unfocused. But it does get right that this is not a trivial thing. I think we'd indeed need to export two versions. I'd like to wait until the ecosystem comes along a bit, then migrate our entire stack to v4, use that as the single source of truth, and generate v3 versions and export those separately. I'd like to do that for a v5 of this module. We should look up percentages if we can (hard because both npm versions ship both code versions) but just from vibing on x, not a lot of people on v3 have migrated to v4 just yet.

For those that did, or had fresh projects now on v4 already, I think the advice for now should be to use the /v3 inside their projects if they need to change transloadit schemas.

If they don't have a use case for changing the schemas.. perhaps they don't even need to care that we validate our responses with zod and infer our types from it. They can just use zod4 in their project for our own things, use our types as they would if we had been on it already, and they can even use our schemas in isolation to validate some data.

What do you think about this plan @Murderlon @remcohaszing @mifi ?

kvz avatar Nov 04 '25 15:11 kvz

I'm not sure what happens now with consumers using v4, does it partially work? all errors? I think we should partially base our decision on that. I'm also worried that maintaining two versions is not trivial and we'll potentially end up with lots duplication to please one version or the other. All in all, I agree it's best to not spread our time too thin and just flip the switch at once instead of migrating the SDK twice (first dual, then v4 only).

Murderlon avatar Nov 05 '25 08:11 Murderlon

What compatibility issues are users running into?

Maintaining a Zod 3 and a Zod 4 version seems like a lot of effort for no added benefit.

remcohaszing avatar Nov 05 '25 08:11 remcohaszing

Iiuc, someone on zod4 can perfectly use our exported v3 schema and use it to validate some things. It’s a selfcontained thing and v3 and v4 can coexist.

I think you only run into problems when you try to extend a zod4 schema with our v3 one, and so on.

Should be a quite niche use case that we can also explicitly recommend against: Tie in with our types, not with our schemas.

I short I’m with Remco wanting to hear what exact problem people are hitting. Likely there is a better solution/workaround for it than dualwielding all schemas at this point in time at least.

kvz avatar Nov 05 '25 09:11 kvz

Potentially https://x.com/colinhacks/status/1991793959836758406?s=20 / https://github.com/modelcontextprotocol/typescript-sdk/pull/1040 can teach us something here

kvz avatar Nov 21 '25 10:11 kvz

Image

https://x.com/colinhacks/status/1991969016983019754

https://zod.dev/library-authors

kvz avatar Nov 21 '25 20:11 kvz

I did this now by generating @transloadit/zod/v4 from the source of truth: @transloadit/zod/v3. The schemas of the latter are synced with our internal repos that are still on v3. This makes it work internally, and for consumers already on v4.

We can now take our time to upgrade to v4 internally. That said, since some things in v4 are impossible to express in v3, I wonder if we should do that if most people have upgraded to v4 already in the ecosystem. So that there is less need for an up to date generated v3, that sooner or later will drift from v4 because we'll use a thing that it can't support.

I think for now we can close this issue, all consumer should be satisfied I expect. And when Transloadit thinks the time is right to switch internally, we can revisit.

kvz avatar Jan 21 '26 19:01 kvz