RFC: 18 Open context provider framework

[ddneilson]

---

title: RFC: #18 Open context provider framework labels: management/rfc

Rendered version

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[mergify[bot]]

Title does not follow the guidelines of Conventional Commits. Please adjust title before merge.

[ddneilson]

I appreciate the feedback, @rix0rrr . I'm trying to find some time to digest your comments, and dig back into this; I'll aim to have something up within the next week or so.

[john-tipper]

For me, the concrete use case I wish to use this feature for is in resolving multiple accounts for complicated cross-account deployments. We have a lot of AWS Organization accounts, which are identifiable by name, tag etc. I want the CDK to be able to resolve which accounts exist and which stacks need to go into which account at synth time. This will allow me to do things like update IAM resource policies to permit access from new accounts really simply.

I'd like to KISS and be able to do something simple, like we do for credentials, and supply the plugin explicitly on the command line. My implementation of that plugin could then return context (via whatever mechanism my plugin uses: SDK call, REST call, bash script etc) that might give a list of accounts that meet some criteria (e.g. being tagged with some value).

[eladb]

@ddneilson is this something you'd be interested in continuing to work on or should we close for now? (always happy for you to pick this up in the future)

[ddneilson]

@ddneilson is this something you'd be interested in continuing to work on or should we close for now? (always happy for you to pick this up in the future)

@eladb

Interested, yes. Available bandwidth? No.

I would still love to see it possible for libraries other than the core CDK construct library make use of the provider framework. Synth-time values are important to more than just the CDK.

For instance, we have a construct that works best if we know the account's ELB Limits at synth time -- https://github.com/aws/aws-rfdk/blob/f84097ef4c084916b1affbd3de55a8a11bba1d04/packages/aws-rfdk/lib/core/lib/health-monitor.ts#L187. The construct creates one or more load balancers to run Health Checks, but has to take into account the maximum number of instances a single LB can monitor (which is an account limit) to know how many LBs to create. The only way to get that right now is to ask the customer to query the limits themselves, and manually enter them into their app. It's a horrid customer experience.

[chris-smith-zocdoc]

One area this would be super useful is having the end user write custom Lookup functions for capabilities that aren't bundled with the cdk.

There are several open issues requesting various lookups that could be written by the end user if the context provider framework was extensible

https://github.com/aws/aws-cdk/issues/8461 https://github.com/aws/aws-cdk/issues/6803 https://github.com/aws/aws-cdk/issues/14032 https://github.com/aws/aws-cdk/issues/1417

Currently the "best practice" as defined by the docs say

If you need some value (from AWS or elsewhere) for which there is no native CDK context provider, we recommend writing a separate script to retrieve the value and write it to a file, then read that file in your CDK app. Run the script only when you want to refresh the stored value, not as part of your regular build process.

which is less than ideal from a workflow perspective

[sciutand]

I just came across this. My issue is that I want to use some instance type that are not available in all az. I receive the instance type as a parameter and there is no way for me to find out which az's the instance is available for the given region/account combination. And I need to pass that to my autoscaling group or it will try to launch an instance in the wrong az and fail miserably. Upvote on allowing custom providers.

[RichiCoder1]

Howdy! I know there has been work and there's "sort of" support for this in the CDK right now, but I was wondering if this was still something that was in progress?

If not, something external contributors could help move forward?

[marcogrcr]

My use case is that I want to create a DynamoEventSource for subscribing a Lambda function for each replica of a global table. However CfnGlobalTable.attrStreamArn and Table.tableStreamArn only return the ARN of the stream for the table in the region of the stack that contains the resource and there's no way to automatically obtain the stream ARNs of the replicas without a CFN custom resource or a custom script.

This, with this feature I could create this instead:

function lookupTableByName(scope: Construct, name: string): ITable

Which uses a context provider plugin that internally invokes dynamodb:DescribeTable.

[marcogrcr]

@RichiCoder1 it seems the current state of ContextProviderPlugin is very rough:

Users can author plugins as follows:

interface MyPluginQuery {
  /*
   * These fields will be automatically populated using the stack's account/region if the `ContextProvider.getValue()`
   * invocation omits the `includeEnvironment` param field or sets it to `true`.
   */
  readonly account: string;
  readonly region: string;

  // custom fields
  readonly myField: unknown;
}

interface MyPluginContext {
  // custom fields
  readonly myField: unknown;
}

class MyPlugin implements ContextProviderPlugin {
  public async getValue(args: MyPluginQuery): Promise<MyPluginContext> {
    // ...
  }
}

export = {
  version: "1",
  init(host: PluginHost) {
    host.registerContextProviderAlpha("my-plugin", new MyPlugin());
  }
} as Plugin;

And then use them as follows:

type MyPluginProps = Omit<MyPluginQuery, "account" | "region">;

function myPluginLookup(scope: Construct, props: MyPluginProps): MyPluginContext {
  return ContextProvider.getValue(scope, {
    provider: "plugin",
    props: {
      ...props,
      pluginName: "my-plugin"
    },
    includeEnvironment: true, // so account/region are populated from `scope`'s stack
    dummyValue: { myField: "..." },
  }).value;
}

Where:

• ContextProvider is defined here.
• ContextProviderPlugin is defined here.
• Plugin is defined here.
• PluginHost is defined here.

However, the following limitations apply:

• None of the types (except ContextProvider) are exported by the corresponding npm packages: users will have to use any or copy/paste the type defintions.
• The plugin cannot receive an SdkProvider instance in the constructor: Unlike the built-in context providers (e.g. SSMContextProviderPlugin), context provider plugins cannot re-use the AWS credentials configured via CDK nor CredentialProviderSource plugins to obtain an instance of an aws-sdk client pre-configured with the appropriate credentials. Even if SdkProvider where accessible by plugins, it provides a limited subset of the clients.
• Only intended to be used internally: Judging by this documentation it seems like this feature is only meant to be used internally for now, though it seems to be under active development (e.g. https://github.com/aws/aws-cdk/pull/18709).