Serverless V4 Upgrade: unable to locate job scripts

[josh-green-awaze]

Issue

I'm going through the process of upgrading to Serverless V4 and I've come across an issue with serverless-glue that wasn't happening with V3. We're using the Compose functionality of Serverless where the file structure is:

- root/
  - serverless-compose.yml
  - Utils/
    - serverless.yml
    - glue-scripts/
      - script.py

Previously in our serverless.yml we had the glue job defined as:

Glue:
  bucketDeploy: ${self:custom.Glue.BucketName}
  createBucket: true
  createBucketConfig:
    ACL: private
  s3Prefix: scripts/
  tempDirBucket: ${self:custom.Glue.BucketName}
  tempDirS3Prefix: temporary/
  jobs:
    - name: glue-job-name
      scriptPath: glue-scripts/script.py
...

but upon deploy with Serverless V4 it started erroring with: ENOENT: no such file or directory, open 'glue-scripts/script.py'.

The code around reading the files hasn't changed, so I'm guessing something about the Serverless V4 upgrade has broken this: https://github.com/toryas/serverless-glue/blob/master/src/services/serverless.service.ts#L124

Work Around

By changing the scriptPath to be absolute from the root it works:

  jobs:
    - name: glue-job-name
      scriptPath: Utils/glue-scripts/script.py

I'm guessing that Serverless have started running their compose or plugins in a different cwd, but I can't find mention of it in their upgrade notes.