serverless-step-functions

serverless-step-functions copied to clipboard

Serverless Version: 1.78.1 Plugin Version: 2.22.1

Let's have a Typescript types for serverless-step-functions

Why

1. ts-check fails when using serverless-step-functions because stepFunctions is not expected by Serverless defintion
2. Types would make work quicker and easier in VS Code and other IDEs that support Typescript
3. It would also give human readers a good overview of stepFunctions

What

• Something like this definition for Serverless from the @types/serverless npm

Notes

I have started doing this in my own project. See an example for my specific use case below

I'm raising this issue to see where/how/if we can develop a complete definition collectively.

// serverless.ts
import { Serverless } from "serverless/aws";
import { contentfulEnvironmentVariables } from "./src/config/contenful";

type Definition = {
  Comment?: string;
  StartAt: string;
  States: {
    [state: string]: {
      Catch?: Catcher[];
      Type: "Map" | "Task" | "Choice" | "Pass";
      End?: boolean;
      Next?: string;
      ItemsPath?: string;
      ResultPath?: string;
      Resource?: string | { "Fn::GetAtt": string[] };
      Iterator?: Definition;
    };
  };
};

type Catcher = {
  ErrorEquals: ErrorName[];
  Next: string;
  ResultPath?: string;
};

type ErrorName =
  | "States.ALL"
  | "States.DataLimitExceeded"
  | "States.Runtime"
  | "States.Timeout"
  | "States.TaskFailed"
  | "States.Permissions"
  | string;

interface ServerlessWithStepFunctions extends Serverless {
  stepFunctions: {
    stateMachines: {
      [stateMachine: string]: {
        name: string;
        definition: Definition;
      };
    };
    activities?: string[];
    validate?: boolean;
  };
}

// example config (in a Typescript Serverless project this can be used in place of serverless.yml)

const serverlessConfiguration: ServerlessWithStepFunctions = {
  service: {
    name: "xxx",
  },
  frameworkVersion: ">=1.72.0",
  custom: {
    webpack: {
      webpackConfig: "./webpack.config.js",
      includeModules: true,
    },
  },
  plugins: ["serverless-step-functions", "serverless-webpack"],
  provider: {
    name: "aws",
    region: "eu-west-1",
    runtime: "nodejs12.x",
    timeout: 60,
    apiGateway: {
      minimumCompressionSize: 1024,
    },
    environment: {
      AWS_NODEJS_CONNECTION_REUSE_ENABLED: "1",
    },
  },
  functions: {
    createImageInContentful: {
      handler: "src/titles/createImage.handler",
      description: "Create an image asset in Contentful",
      environment: { ...contentfulEnvironmentVariables },
    },
    publishImageInContentful: {
      handler: "src/titles/publishImage.handler",
      description: "Publish an image asset in Contentful",
      environment: { ...contentfulEnvironmentVariables },
    },
  },
  stepFunctions: {
    stateMachines: {
      migrateAllTitlesMainImage: {
        name: "MigrateAllTitlesMainImage",
        definition: {
          Comment: "Migrate images from titles from Airtable into Contentful",
          StartAt: "MigrateAll",
          States: {
            MigrateAll: {
              Type: "Map",
              End: true,
              ItemsPath: "$.titles",
              Iterator: {
                StartAt: "CreateContentfulAsset",
                States: {
                  CreateContentfulAsset: {
                    Type: "Task",
                    Next: "PublishContentfulAsset",
                    Resource: {
                      "Fn::GetAtt": ["createImageInContentful", "Arn"],
                    },
                    Catch: [
                      {
                        ErrorEquals: ["VersionMismatch"],
                        ResultPath: "$.CreateContentfulAssetError",
                        Next: "AssetAlreadyCreated",
                      },
                    ],
                    ResultPath: "$.CreateContentfulAssetResult",
                  },
                  AssetAlreadyCreated: {
                    Type: "Pass",
                    Next: "PublishContentfulAsset",
                  },
                  PublishContentfulAsset: {
                    Type: "Task",
                    End: true,
                    Resource: {
                      "Fn::GetAtt": ["publishImageInContentful", "Arn"],
                    },
                  },
                },
              },
            },
          },
        },
      },
    },
    activities: ["content-migration-titles-images"],
    validate: true,
  },
};

module.exports = serverlessConfiguration;

@LL782 I'm confused by the use case here - is it to make it easier for someone else to work on this plugin? Do the type definitions surface in the serverless.yml (via some VS Code plugin)?

@theburningmonk thanks for asking for clarity. Yes the definitions are there for that reason and a couple of others.

1. Prompts and tips are surfaced when editing stepFunctions in the serverless configuration
2. Human readers familiar with Typescript can use them for guidance
3. When using Typescript we have to define something for stepFunctions otherwise the Typescript check will fail

I've added more details to the description above. Hope that helps

Issue description updated

Just FYI, I had a quick thought that we could refer type definition from aws-cdk typescript to implement what is proposed by LL782, but it didn't work. They have rather high-level object suitable for manipulation to build up step function definition than definition of step function JSON itself.

For instance, Condition is just a class with static properties which doesn't represent its structure.

@LL782 I find defining Step Functions can be quite fiddly so a type definition for this awesome package would be really useful.

Building on this example it might be good to define the different step types with separate types

type Step = {
  End?: boolean
  Next?: string
  ItemsPath?: string
  ResultPath?: string
  Resource?: string | { 'Fn::GetAtt': string[] }
  Catch?: Catcher[]
}

interface Task extends Step {
  Type: 'Task'
}

interface Map extends Step {
  Type: 'Map'
  Iterator: Definition
}

interface Choice extends Step {
  Type: 'Choice'
  Choices: any[] //TODO define Choices
}

type Pass = {
  Type: 'Pass'
  End?: boolean
  Next?: boolean
}

type Definition = {
  Comment?: string
  StartAt: string
  States: {
    [state: string]: Task | Map | Choice | Pass
  }
}

@toddpla this is a great development. In fact we went on to do a similar thing on the project I was on

type StateMachines = {
  [stateMachine: string]: {
    name: string;
    definition: Definition;
  };
};

type Definition = {
  Comment?: string;
  StartAt: string;
  States: States;
};

type States = {
  [state: string]: Choice | Fail | Map | Task | Parallel | Pass | Wait;
};

type StateBase = {
  Catch?: Catcher[];
  Retry?: Retrier[];
  End?: boolean;
  InputPath?: string;
  Next?: string;
  OutputPath?: string;
  ResultPath?: string;
  ResultSelector?: { [key: string]: string | { [key: string]: string } };
  Type: string;
};

interface Choice extends StateBase {
  Type: "Choice";
  Choices: ChoiceRule[];
  Default?: string;
}

interface Fail extends StateBase {
  Type: "Fail";
  Cause?: string;
  Error?: string;
}

interface Map extends StateBase {
  Type: "Map";
  ItemsPath: string;
  Iterator: Definition;
}

type Resource =
  | string
  | { "Fn::GetAtt": [string, "Arn"] }
  | { "Fn::Join": [string, Resource[]] };

interface TaskParametersForLambda {
  FunctionName?: Resource;
  Payload?: {
    "token.$": string;
    [key: string]: string;
  };
  [key: string]: unknown;
}

interface TaskParametersForStepFunction {
  StateMachineArn: Resource;
  Input?: {
    "AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID.$"?: "$$.Execution.Id";
    [key: string]: string;
  };
  Retry?: [{ ErrorEquals?: string[] }];
  End?: boolean;
}

interface Task extends StateBase {
  Type: "Task";
  Resource: Resource;
  Parameters?:
    | TaskParametersForLambda
    | TaskParametersForStepFunction
    | { [key: string]: string | { [key: string]: string } };
}

interface Pass extends StateBase {
  Type: "Pass";
  Parameters?: {
    [key: string]: string | Array<unknown> | { [key: string]: string };
  };
}

interface Parallel extends StateBase {
  Type: "Parallel";
  Branches: Definition[];
}

interface Wait extends StateBase {
  Type: "Wait";
  Next?: string;
  Seconds: number;
}

type Catcher = {
  ErrorEquals: ErrorName[];
  Next: string;
  ResultPath?: string;
};

type Retrier = {
  ErrorEquals: string[];
  IntervalSeconds?: number;
  MaxAttempts?: number;
  BackoffRate?: number;
};

type ErrorName =
  | "States.ALL"
  | "States.DataLimitExceeded"
  | "States.Runtime"
  | "States.Timeout"
  | "States.TaskFailed"
  | "States.Permissions"
  | string;

Perhaps we should look at contributing to DefinitelyTyped...awsProvider.d.ts

I'm no longer on the project where we developed the definitions above so it would be difficult for me to assure the quality of it meets DefintielyTyed guidelines but I'd be happy to get the ball moving if others are ~~interested~~ willing to support.

Sounds like a good idea. Happy to support. 😄

No ChoiceRule defined :)

No ChoiceRule defined :)

Ah true. Good spot @deser 🙌

I've moved on from the project where I was using ServerlessJS no longer have access to the codebase where I was working out these types.

If you or anyone can define ChoiceRule I'll update my comment above 🙂

@deser @LL782 I added a ChoiceRule type and some other fields. The whole definition is available in this gist: https://gist.github.com/zirkelc/084fcec40849e4189749fd9076d5350c

Here's the type again so you can update your comment:

type ChoiceRuleComparison = {
  Variable: string;
  BooleanEquals?: number;
  BooleanEqualsPath?: string;
  IsBoolean?: boolean;
  IsNull?: boolean;
  IsNumeric?: boolean;
  IsPresent?: boolean;
  IsString?: boolean;
  IsTimestamp?: boolean;
  NumericEquals?: number;
  NumericEqualsPath?: string;
  NumericGreaterThan?: number;
  NumericGreaterThanPath?: string;
  NumericGreaterThanEquals?: number;
  NumericGreaterThanEqualsPath?: string;
  NumericLessThan?: number;
  NumericLessThanPath?: string;
  NumericLessThanEquals?: number;
  NumericLessThanEqualsPath?: string;
  StringEquals?: string;
  StringEqualsPath?: string;
  StringGreaterThan?: string;
  StringGreaterThanPath?: string;
  StringGreaterThanEquals?: string;
  StringGreaterThanEqualsPath?: string;
  StringLessThan?: string;
  StringLessThanPath?: string;
  StringLessThanEquals?: string;
  StringLessThanEqualsPath?: string;
  StringMatches?: string;
  TimestampEquals?: string;
  TimestampEqualsPath?: string;
  TimestampGreaterThan?: string;
  TimestampGreaterThanPath?: string;
  TimestampGreaterThanEquals?: string;
  TimestampGreaterThanEqualsPath?: string;
  TimestampLessThan?: string;
  TimestampLessThanPath?: string;
  TimestampLessThanEquals?: string;
  TimestampLessThanEqualsPath?: string;
};

type ChoiceRuleNot = {
  Not: ChoiceRuleComparison;
  Next: string;
};

type ChoiceRuleAnd = {
  And: ChoiceRuleComparison[];
  Next: string;
};

type ChoiceRuleOr = {
  Or: ChoiceRuleComparison[];
  Next: string;
};

type ChoiceRuleSimple = ChoiceRuleComparison & {
  Next: string;
};

type ChoiceRule = ChoiceRuleSimple | ChoiceRuleNot | ChoiceRuleAnd | ChoiceRuleOr;

interface Choice extends StateBase {
  Type: 'Choice';
  Choices: ChoiceRule[];
  Default?: string;
}

@horike37 I would like to submit a PR with Typescript definitions if you are interested to include them with the package?

This would be really helpful. Is not yet implemented in any way?

@ebisbe as far as I know it hasn't gone any further than this issue. There is a lot of really useful information in here though.

Personally I haven't had opportunity to work on a serious step functions project for a year or two now, which is why I haven't worked on implementation myself. I'm happy to support if I can and you want take it forward. I'm sure you'd get a lot of support from people in this conversation if you want to move with it.

I submitted PR https://github.com/serverless-operations/serverless-step-functions/pull/585

Would be nice to double check with you guys if it works for you. You can install the branch directly with NPM, Yarn, or PNPM and see if the types appear:

pnpm add -D zirkelc/serverless-step-functions#typescript-types

We created type definitions for this package: https://www.npmjs.com/package/@types/serverless-step-functions

You can use them like this in your serverless.ts config:

import type { AWS as Serverless } from '@serverless/typescript';
import type StepFunctions from 'serverless-step-functions';

declare module '@serverless/typescript' {
  interface AWS {
    stepFunctions?: StepFunctions;
  }
}

const serverless: Serverless = {
  service: 'nebula-connector-master',
  frameworkVersion: '3',
  plugins: ['serverless-esbuild', 'serverless-step-functions'],
  provider: {
    name: 'aws',
    runtime: 'nodejs16.x',
    region: 'eu-west-1',
    stage: 'dev',
    timeout: 30,
  },
  functions: {
    hello: {
      handler: 'src/functions/hello/handler.hello',
    },
  },
  stepFunctions: {
    stateMachines: {
      hellostepfunc1: {
        name: 'myStateMachine',
        definition: {
          Comment: 'A Hello World example of the Amazon States Language using an AWS Lambda Function',
          StartAt: 'HelloWorld1',
          States: {
            HelloWorld1: {
              Type: 'Task',
              Resource: {
                'Fn::GetAtt': ['hello', 'Arn'],
              },
              End: true,
            },
          },
        },
        dependsOn: ['CustomIamRole'],
        tags: {
          Team: 'Atlantis',
        },
      },
    },
    validate: true,
    noOutput: false,
  },
};

module.exports = serverless;

There is an open issue for the @serverless/typescript package: https://github.com/serverless/typescript/issues/82

When this issue is resolved, we can use module augmentation to automatically extend the types @serverless/typescript with the types from serverless-step-functions without the need to add declare module ... at the beginning:

declare module '@serverless/typescript' {
  interface AWS {
    stepFunctions?: StepFunctions;
  }
}