Some paths not resolved...

[QAnders]

First off!

Thanks so much for fixing this! I did a "clone" previously (https://www.npmjs.com/package/serverless-offline-aws-ssm-local) that has been working well up until the latest release of Serverless so I was digging for something new and updated and found your package! Very much appreciated!

However I have run into a bit of a snag that I haven't figured out... Some variables don't get resolved.

I am starting the Serverless offline with -s dev and have provider: stage: dev in my serverless.yml so running it as dev. I am using all the latest versions of serverless and plugins.

serverless.yml

plugins:
  - serverless-offline-ssm
  - serverless-offline
  - serverless-prune-plugin

The custom section (I don't use any .env):

 custom:
  prune:
    automatic: true
    number: 10
  serverless-offline:
    httpPort: 3001
  serverless-offline-ssm:
    # NB! The offline variables will get pushed to GitHub so NEVER include any online/AWS credentials here!
    stages:
     - dev
    ssm:
      '/dev/lambda/common/LAMBDA_STAGE': dev
      '/dev/lambda/common/SENTRY_DSN': https://xxx:[email protected]/160000
      '/dev/lambda/common/LOGGLY_TOKEN': http://loggly.com.fake/'
      '/dev/lambda/common/VPC_SECURITY_GROUP_ID': sg-xxx
      '/dev/lambda/common/VPC_SUBNET_ID1': subnet-xxx
      '/dev/lambda/common/VPC_SUBNET_ID2': subnet-xxx
      '/dev/lambda/common/AWS_S3_ENDPOINT': https://s3-eu-west-1.amazonaws.com
      '/dev/lambda/common/PG_HOST': localhost
      '/dev/lambda/common/PG_USER': dbuser
      '/dev/lambda/common/PG_PASSWORD': password
      '/dev/lambda/common/PG_DATABASE': db
      '/dev/lambda/common/PG_PORT': 5432
      '/dev/lambda/common/PROXY_QIP_HOST': http://localhost:1337
      '/dev/lambda/common/RDS_AURORA_DATA_API_SECRET_ARN': arn:aws:secretsmanager:eu-west-1:2900000000:secret:dummy
      '/dev/lambda/common/RDS_AURORA_DATA_API_ARN': arn:aws:rds:eu-west-1:2900000000:cluster:dummy
      '/dev/lambda/common/RDS_AURORA_DATA_API_QIP_DB': db

And adding env.var's as this:

environment:
    LAMBDA_STAGE: ${ssm:/${opt:stage, self:provider.stage}/lambda/common/LAMBDA_STAGE}
    SENTRY_DSN: ${ssm:/${opt:stage, self:provider.stage}/lambda/common/SENTRY_DSN}
    LOGGLY_TOKEN: ${ssm:/${opt:stage, self:provider.stage}/lambda/common/LOGGLY_TOKEN}
    RDS_AURORA_DATA_API_SECRET_ARN: ${ssm:/${opt:stage, self:provider.stage}/lambda/common/RDS_AURORA_DATA_API_SECRET_ARN}

When firing up serverless offline I get this in the startup messages:

Serverless: serverless-offline-ssm checking serverless version 2.40.0.
Serverless: Deprecation warning: Variables resolver reports following resolution errors:
              - Cannot resolve variable at "provider.environment.RDS_AURORA_DATA_API_SECRET_ARN": Value not found at "ssm" source,
              - Cannot resolve variable at "provider.environment.RDS_AURORA_DATA_API_ARN": Value not found at "ssm" source,
              - Cannot resolve variable at "provider.environment.RDS_AURORA_DATA_API_QIP_DB": Value not found at "ssm" source,
              - Cannot resolve variable at "provider.environment.PG_PORT": Value not found at "ssm" source
            From a next major this will be communicated with a thrown error.
            Set "variablesResolutionMode: 20210326" in your service config, to adapt to new behavior now
            More Info: https://www.serverless.com/framework/docs/deprecations/#NEW_VARIABLES_RESOLVER

What I then noticed is that the variables it can't resolve are not created in AWS SSM and that all variable values are picked from AWS online and not my offline var's.

What have I missed?

[QAnders]

Ugh... Shoul'da done some loggin first...

Did some more testing and found that the variables that exists in AWS SSM (i.e. "online") is used if they exist, so e.g., /dev/lambda/common/PG_USER that exists in AWS SSM will give me the value of db_admin which is the "real" value and not the value dbuser expected from the offline SSM var's.

However the variable /dev/lambda/common/RDS_AURORA_DATA_API_SECRET_ARN that doesn't exist in AWS SSM will give me the value I have set in offline in serverless.yml, but then with the Serverless warning Cannot resolve variable at "provider.environment.RDS_AURORA_DATA_API_SECRET_ARN"

Is this intended?

I can get "round" it by using a different stage locally of course (e.g. local) but I would still have that Serverless warning and from that message it states that:

From a next major this will be communicated with a thrown error.
            Set "variablesResolutionMode: 20210326" in your service config, to adapt to new behavior now
            More Info: https://www.serverless.com/framework/docs/deprecations/#NEW_VARIABLES_RESOLVER

THis would then mean that from the next Serverless major (3.0.0?) it won't work unless the variables are actually in AWS SSM and then there is no way to use local variables anymore... right?

[QAnders]

Some more info... sorry about long thread...

I changed to using local stage and it all runs fine, but as before I get the Serverless warning. So I added the suggested variablesResolutionMode: 20210326 into serverless.yml and tried again and then Serverless Offline fails at startup:

Serverless Error ----------------------------------------

  Cannot resolve serverless.yml: Variables resolution errored with:
    - Cannot resolve variable at "provider.vpc.securityGroupIds.0": Value not found at "ssm" source,
    - Cannot resolve variable at "provider.vpc.subnetIds.0": Value not found at "ssm" source,
    - Cannot resolve variable at "provider.vpc.subnetIds.1": Value not found at "ssm" source,
    - Cannot resolve variable at "provider.environment.LAMBDA_STAGE": Value not found at "ssm" source,

And it then terminates with a npm error

[QAnders]

The "problem" is that Servless itself overwrites the variables from AWS SSM. I added a console.log() to both packages and it logs like:

serverless-offline-ssm reading variables
Serverless: serverless-offline-ssm checking serverless version 2.40.0.
Serverless reading SSM

As Serverless reading SSM happens after and the variables exist they are fetched from online and replaced.

I checked the code and can't figure out a way to circumvent this and from v.3.0.0 it won't work anymore... :(

I've added an issue at Serverless: https://github.com/serverless/serverless/issues/9460

[MelvinVermeer]

I'm experiencing the same issue variablesResolutionMode: 20210326 is basically blocking this plugin.

[QAnders]

https://github.com/serverless/serverless/issues/9460#issuecomment-839918367

[tschleuss]

have you guys found a temporary workaround on this? I'm having the same issue

[tschleuss]

I created a temporary plugin to bypass this for now based on other plugins that I saw online that aren't working properly. This one is working with variablesResolutionMode: 20210326

/plugins/offline-ssm/index.js

class OfflineSSM {
  constructor(serverless, options) {
    this.serverless = serverless;
    this.service = serverless.service;
    this.config = (this.service.custom && this.service.custom.offlineSSM) || {};
    this.profile = serverless.service.provider.profile;

    if (!this.shouldExecute()) {
      return;
    }

    const aws = this.serverless.getProvider('aws');
    const originalRequest = aws.request.bind(aws);

    aws.request = (service, method, params) => {
      if (service !== 'SSM' || method !== 'getParameter') {
        return originalRequest(service, method, params, options);
      }
      const { Name } = params;
      const Parameter = this.config.ssm[Name];
      return Promise.resolve({ Parameter });
    };

    this.serverless.setProvider('aws', aws);
  }

  shouldExecute() {
    if (this.config.profiles && this.config.profiles.includes(this.profile)) {
      return true;
    }
    return false;
  }
}

module.exports = OfflineSSM;

serverless.yml

plugins:
   - ./plugins/offline-ssm

offlineSSM: 
  profiles:
    - dev
  ssm:
    /global/MY_SECRET_API_KEY:
      Type: SecureString
      Value: mock-key

Whichever attributes you define below the key name, offline-ssm will return to serverless. I had to do this way because in my case serverless was expecting a key with Type : SecureString, you can define all the other responses if you want

      "ARN": "string",
      "DataType": "string",
      "LastModifiedDate": number,
      "Name": "string",
      "Selector": "string",
      "SourceResult": "string",
      "Type": "string",
      "Value": "string",
      "Version": number

This plugin will just trigger based on the provider.profile attribute, but you can change if you want to to use stage or any other.

[rs-brandon]

I made some modifications for grabbing from the serverless object - I think the schema is slightly different in my version of SSM.

/** Mostly taken from conversation around this GitHub issue on the serverless-offline-ssm plugin.
*   https://github.com/janders223/serverless-offline-ssm/issues/145
*/

const OFFLINE_STAGE = 'local';

class OfflineSSM {
    constructor(serverless, options) {
        this.serverless = serverless;
        this.service = serverless.service;

        this.config = (this.serverless.configurationInput.custom && this.serverless.configurationInput.custom['offline-ssm']) || {};
        this.options = options

        if (!this.shouldExecute()) {
            return;
        }

        const aws = this.serverless.getProvider('aws');
        const originalRequest = aws.request.bind(aws);

        aws.request = (service, method, params) => {
            if (service !== 'SSM' || method !== 'getParameter') {
                return originalRequest(service, method, params, options);
            }
            const { Name } = params;
            const Parameter = this.config.ssm[Name];
            return Promise.resolve({ Parameter });
        };

        this.serverless.setProvider('aws', aws);
    }

    shouldExecute() {
        if (this.options.stage == OFFLINE_STAGE) {
            return true;
        }
        return false;
    }
}

module.exports = OfflineSSM;