Support for more complex JsonPath

[redvex]

The StepFunctions implementation of JsonPath allow us to filter elements using the ?() syntax. When I try to test a state machine that use a filter, the validation fails and if I try to bypass the validation, the engine returns an error.

Step to reproduce

import { StateMachine } from 'aws-local-stepfunctions';

const machineDefinition = JSON.parse(JSON.stringify({
  StartAt: 'GetSubjects',
  States: {
    GetSubjects: {
      Type: 'Task',
      Resource: 'arn:aws:lambda:us-east-1:123456789012:function:GetSubjects',
      Next: 'ReturnResult'
    },
    ReturnResult: {
      Type: 'Pass',
      Parameters: {
        'completed_los.$': '$.Payload[?(@.subject_status==subject_completed)]..subject_id',
      },
      End: true
    }
  },
}));

test('OK', async () => {
  // GIVEN
  const myInput = {};

  // WHEN
  const stateMachine = new StateMachine(machineDefinition, {
    // validationOptions: {
    //   checkPaths: false,
    // }
  });
  const execution = stateMachine.run(myInput, {
    overrides: {
      taskResourceLocalHandlers: {
        "GetSubjects": ((event: any) => {
          return {
            Payload: [
              { subject_id: 1, subject_status: 'subject_completed' },
              { subject_id: 2, subject_status: 'subject_completed' },
              { subject_id: 3, subject_status: 'subject_added' }
            ]
          }
        })
      },
    },
  });
  const result = await execution.result;
  console.log(result);

  // THEN
  expect(result).toStrictEqual([1,2]);
});

Desired outcome

The state machine should process the JsonPath filter.

Error

tate machine definition is invalid, see error(s) below:
     SCHEMA_VALIDATION_FAILED: /States/ReturnResult/Parameters/completed_los.$ is invalid. must match format "asl_payload_template"

By setting checkPaths: false, the state machine is executed, but I get the following error:

ExecutionError: Execution has failed with the following error: jsonPath: subject_completed is not defined: @.subject_status==subject_completed

By changing the definition to wrap the condition with double quotes $.Payload[?(@.subject_status=="subject_completed")]..subject_id, it works, but unfortunately that doesn't work on step function:

Workaround

To make it work, I could pre-process the state machine definition to double quote any jsonpath expression, but that would be quite a good effort, moreover, I'm not keen to disable validations.

[nibble-4bits]

Hi @redvex,

This is an issue that has to do with the JSONPath library used in this project, which is jsonpath-plus. It seems this kind of syntax is not supported by that library. I too have noticed some path expressions which are valid in the AWS implementation do not work as expected in aws-local-stepfunctions, and vice versa.

Unfortunately, since JSONPath is not standardized, libraries tend to differ in its implementation. There's an RFC proposal to standardize the JSONPath spec, but that's still a draft.