amplify-category-api icon indicating copy to clipboard operation
amplify-category-api copied to clipboard

Migrate from VTL Templates to APPSYNC_JS

Open cheruvian opened this issue 4 months ago β€’ 5 comments

Describe the feature you'd like to request

Currently, Amplify API generates all resolvers using VTL (Velocity Template Language). This has a couple drawbacks:

  • Niche language: VTL has minimal adoption outside AppSync and legacy Java apps. Most developers have no prior exposure to it.
  • Custom AppSync extensions: AppSync adds custom VTL utilities ($util.dynamodb, $util.transform, etc.) not found in standard VTL documentation.
  • Poor debugging: VTL errors are cryptic, there's no step-through debugging, and the feedback loop is slow.
  • Difficult to extend: Customizing auto-generated resolvers requires learning VTL's quirksβ€”inconsistent null handling, unusual syntax, lack of modern language features.
  • Weak tooling: Minimal IDE support, no autocomplete, no type checking.
  • Hard to test: Unit testing VTL in isolation is non-trivial compared to JavaScript.

AppSync now officially supports APPSYNC_JS as a first-class runtime. JavaScript is nearly universal among web/mobile developers, has excellent tooling, and resolvers can be tested with standard tools like Jest.

Describe the solution you'd like

Update the GraphQL transformer to generate APPSYNC_JS resolvers instead of VTL. This should be the new default for all generated resolvers including: pipeline resolvers, unit resolvers, and custom business logic.

Example of the improvement:

VTL (current):

#set($modelQueryExpression.expression = "#typeName = :typeName")
#set($modelQueryExpression.expressionNames = { "#typeName": "__typename" })
#set($modelQueryExpression.expressionValues = {
  ":typeName": $util.dynamodb.toDynamoDB($ctx.args.typeName)
})
#if(!$util.isNull($ctx.args.sortDirection) && $ctx.args.sortDirection == "DESC")
  #set($modelQueryExpression.scanIndexForward = false)
#else
  #set($modelQueryExpression.scanIndexForward = true)
#end

APPSYNC_JS (proposed):

import { util } from '@aws-appsync/utils';

export function request(ctx) {
  const { typeName, sortDirection } = ctx.args;
  return {
    operation: 'Query',
    query: {
      expression: '#typeName = :typeName',
      expressionNames: { '#typeName': '__typename' },
      expressionValues: util.dynamodb.toMapValues({ ':typeName': typeName }),
    },
    scanIndexForward: sortDirection !== 'DESC',
  };
}

Describe alternatives you've considered

  1. Keep VTL as an option: Could offer both runtimes, but this adds maintenance burden and fragments the community/documentation. Better to move forward with JS.

  2. Manual rewrite: Eject resolvers and rewrite in JS manually, but this breaks the Amplify workflow and requires maintaining resolvers outside the Amplify lifecycle.

  3. Custom transformers: Build custom GraphQL transformers that output JS, but this duplicates significant work that Amplify should handle natively.

  4. Continue with VTL: Invest time mastering VTL, but this knowledge has limited transferability outside AppSync and the syntax remains error-prone.

Additional context

  • APPSYNC_JS is the modern choice: AWS added APPSYNC_JS as a first-class runtime specifically because VTL was a pain point. AWS Docs

    We now primarily support the APPSYNC_JS runtime and its documentation. Please consider using the APPSYNC_JS runtime and its guides here.

  • JavaScript is universal: Nearly every Amplify user already knows JS/TS. VTL requires learning a niche language with minimal transferable value.
  • TypeScript support: APPSYNC_JS supports TypeScript for type-safe resolver development with compile-time checks.
  • Testability: JS resolvers can be unit tested with standard tools using the @aws-appsync/utils package.
  • Linting available: @aws-appsync/eslint-plugin provides linting for JS resolvers.
  • Reduce maintenance burden: Maintaining one runtime (JS) is simpler than supporting both VTL and JS long-term.
  • Community alignment: VTL is consistently cited as a major friction point in Amplify API development. Migrating to JS removes a significant barrier to adoption and customization.

Is this something that you'd be interested in working on?

  • [x] πŸ‘‹ I may be able to implement this feature request

Would this feature include a breaking change?

  • [x] ⚠️ This feature might incur a breaking change

cheruvian avatar Dec 14 '25 06:12 cheruvian

Technical Analysis: VTL β†’ APPSYNC_JS Migration Path

After deep-diving into the codebase, here's an analysis of what this migration would involve and a proposed approach.

Current Architecture

The resolver generation system uses an AST-based approach rather than string templates. The key files are:

  • packages/graphql-mapping-template/src/ast.ts - Defines ~25 expression node types (If, IfElse, ForEach, Set, Ref, etc.)
  • packages/graphql-mapping-template/src/print.ts - Renders AST β†’ VTL strings (~255 lines)

All transformers build resolver logic using this AST:

// Example from auth transformer
compoundExpression([
  set(ref(IS_AUTHORIZED_FLAG), bool(false)),
  iff(
    equals(ref('util.authType()'), str(COGNITO_AUTH_TYPE)),
    compoundExpression([
      ...generateStaticRoleExpression(cognitoStaticRoles),
      ...dynamicRoleExpression(ctx, cognitoDynamicRoles, fields),
    ]),
  ),
])

This is then passed to print() which outputs VTL.

The Key Insight

We don't need to rewrite the ~9,000 AST expression usages across the codebase. We write a new printJs.ts that outputs JavaScript instead of VTL from the same AST:

                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                    β”‚   ast.ts    β”‚  ← Same 25 node types (unchanged)
                    β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜
                           β”‚
              β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
              β”‚                         β”‚
       β”Œβ”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”          β”Œβ”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”
       β”‚  print.ts   β”‚          β”‚  printJs.ts   β”‚  ← NEW
       β”‚ (VTL output)β”‚          β”‚  (JS output)  β”‚
       β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Translation Examples

AST Node VTL Output JS Output
iff(pred, expr) #if($pred) ... #end if (pred) { ... }
ifElse(pred, a, b) #if($p) $a #else $b #end if (p) { a } else { b }
forEach(k, coll, exprs) #foreach($k in $coll)...#end for (const k of coll) {...}
set(ref('x'), val) #set($x = val) let x = val;
ref('ctx.args.input') $ctx.args.input ctx.args.input
qref(expr) $util.qr(expr) expr; (side effect)
toJson(expr) $util.toJson(expr) JSON.stringify(expr)

Existing APPSYNC_JS Infrastructure

Good news - the infrastructure already exists:

  1. TransformerResolver already has addJsFunctionToSlot() method
  2. conversation-transformer package uses pure JS resolvers (proves the pattern works)
  3. APPSYNC_JS_RUNTIME constant and runtime detection exist
  4. @aws-appsync/utils and @aws-appsync/utils/dynamodb** provide JS equivalents

Performance: No Penalty

APPSYNC_JS runs inside AppSync's runtime (not Lambda), so:

  • No cold starts - same as VTL
  • No extra cost - included in AppSync pricing
  • Comparable performance - AWS made trade-offs (no try/catch, no async/await) specifically to ensure this

Challenges to Address

  1. Embedded VTL in ref() nodes: Some refs contain VTL-specific syntax:

    ref('util.defaultIfNull($ctx.args.input.id, $util.autoId())')
    ref('ctx.args.input.keySet()')  // Java-style method
    

    These need translation to JS equivalents (?? operator, Object.keys(), etc.)

  2. $foreach.hasNext(): VTL has this built-in; JS needs index-based comparison

  3. Request/Response wrapper: JS resolvers need export function request(ctx) { ... } structure

  4. APPSYNC_JS limitations: No throw (use util.error()), no try/catch, 32KB bundle limit

Validation Strategy

The codebase has 380 test files including exhaustive e2e auth tests (AuthV2ExhaustiveT1A through T3D). These would validate functional equivalence without needing to verify VTL/JS output directly.

Proposed Implementation Phases

  1. Phase 1: Create printJs.ts with core node type translations
  2. Phase 2: Add translation layer for VTL-specific patterns in ref() nodes
  3. Phase 3: Add feature flag to opt-in to JS resolver generation
  4. Phase 4: Run e2e test suite, fix edge cases
  5. Phase 5: Graduate to default (major version bump)

cheruvian avatar Dec 19 '25 07:12 cheruvian

Hi @cheruvian,

Thanks for this detailed proposal! Your approach of creating a printJs.ts to output JavaScript from the existing AST is elegantβ€”it avoids rewriting ~9,000 AST usages.

This relates to #1015 (56 upvotes) requesting AppSync JS resolver support.

Thanks for the thorough analysis! I am requesting the team for inputs here.

pahud avatar Jan 02 '26 17:01 pahud

It's a good idea, and sounds doable. Since as you mentioned yourself the VTL is generated from convenience functions, I'd like to be clear on what the benefits are we expect to materialize before we embark on this.

  • Niche language: I'll give you that.
  • Custom AppSync extensions (AppSync adds custom VTL utilities ($util.dynamodb, $util.transform, etc.) not found in standard VTL documentation): True, but what's the problem? There is documentation for it, and are customers writing this directly? These extensions are also not built into JavaScript?
  • Poor debugging: No step-through debugging and slow feedback loop will be equivalent between VTL and JS: that's a function of the deployment process, not of the language. Error messages... may perhaps be poor, but I've been sufficiently scarred by undefined is not a function that I'm not sure JS is that much better πŸ˜‰ . Some examples would help motivate this.
  • Difficult to extend (Customizing auto-generated resolvers requires learning VTL's quirksβ€”inconsistent null handling, unusual syntax, lack of modern language features.): I don't understand how the bullet caption covers the contents. The motivation doesn't seem to fall under "extensibility", and in any case is extensibility relevant when the JS is being generated?
  • Weak tooling: No IDE support and type checking seems not relevant if you aren't writing the JS/VTL directly in your IDE?
  • Hard to test: Is APPSYNC_JS that much easier to test?

Was this list of benefits perhaps AI generated?

rix0rrr avatar Jan 05 '26 13:01 rix0rrr

Fair point, and yes, I used AI to expand the original list (I see you too elegantβ€” πŸ˜› ), but the underlying issues come from real issues we've run into debugging/patching and/or limiting our usage of AWS Amplify.

tl;dr; I think it more or less boils down to making community/our/my contributions/extensions to the project easier.

The longer story is that we like Amplify's tooling and framework/mental model but my team often encounter significant limitations/bugs (e.g. https://github.com/aws-amplify/amplify-category-api/issues/3364) that push us toward custom lambdas/code or bypassing amplify altogether. We increasingly find ourselves saying we want to contribute back upstream but we already rarely find time to properly document/report the issue/feature request much less learn a whole new language. If we knew VTL better, maybe we'd have contributed fixes directly. But that's kind of the point.

To be fair, perhaps we should know VTL better if we're operating on an Amplify tech stack but I have to imagine the user population would shrink substantially if that's the stance we take.

Responding to your specific questions (again most/all of these tl;dr; into let us/community extend/contribute easier):

  • Niche language / JS universal: More people familiar means more potential contributors
  • Custom AppSync extensions:
    • True, these would need to exist in both runtimes. IIRC this was more about traceability.
    • The AppSync docs imply VTL is less supported/developed ("primarily supported") which is somewhat concerning although I'm sure they would give you a heads up
    • With JS, the utilities could live in an OSS package (or minimally jsdoc for IDE support) we can inspect and step through even locally (admittedly not that important to me)
    • Also hoping it may push more things into a standard JS library and less code generation (haven't really thought through this too much) but would improve dx allowing us to click through the code we're owning.
  • Weak tooling / Hard to Test / Debugging - Totally fair JS leaves much to be desired for its traceability but i still contend it is a superior experience.
    • Would allow us to write / run / test locally resolvers like any other JS code (tbf could probably do this with AppSync VTL if they published their runtime)
    • Could use mostβ„’ standard JS tools e.g. step through debuggers, not sure if that type of tooling exists for VTL.
    • When we're debugging generated resolvers, we're stuck reading VTL in the AppSync console trying to mentally execute it. Our unfamiliarity with the language makes it that much harder to find the root cause.
  • Difficult to extend - Perhaps an advanced use case but there are several times we've found the need to inject behavior into the resolvers. Some of it makes sense to contribute to amplify, other things are likely plugins we would open source (if a plugin system existed), and other yet are fairly bespoke to the use case but for all we'd prefer to write JS rather than VTL.

Anyway, as I revisit this, the core argument is VTL β†’ JS would lower the barrier for community contributions and extensions.

FWIW, I've also been meaning to write a Feature Request for some sort of plugin/middleware framework, so we can intercept/write our own generators/behaviors (e.g. using CloudFront signed cookies instead of the existing never cacheable s3 presigned urls for protected images) which would make JS vs VTL trivial to write/use on our own but the library/tooling's extensibility is probably its own separate discussion

cheruvian avatar Jan 06 '26 05:01 cheruvian

Yeah, it sounds like a good idea to do, especially since it's the way forward. From the documentation, seems like VTL is -- if not deprecated -- at least not recommended anymore.

You're right that the literal VTL ref()s are going to be hardest to translate. We're probably going to have to duplicate them to support the duplicate deployment model:

// Or whatever the appropriate JS syntax would be
ref('util.defaultIfNull($ctx.args.input.id, $util.autoId())', 'context.input.id ?? util.autoId()')

I agree this is going to be a good thing to do. As usual, can't make any promises about timelines. Thanks for the thorough investigation you've done already!

rix0rrr avatar Jan 06 '26 09:01 rix0rrr