aws_kinesisanalytics_flink: Unable to create PropertyGroups in Python

[rgustafson-ie]

Unable to create PropertyGroups to pass to the the Flink Application in Python.

Reproduction Steps

Following the examples in the overview, I would expect something like the following to work:

Using keyword args:

        application = kaf.Application(
            self,
            "FlinkApplication",
            code=kaf.ApplicationCode.from_asset(config.asset_path),
            runtime=kaf.Runtime.FLINK_1_11,
            application_name=f"{construct_id}BeamApplication",
            auto_scaling_enabled=True,
            checkpointing_enabled=True,
            property_groups=kaf.PropertyGroups(
                FlinkApplicationProperties={
                    "input_stream_name": "my-input-kinesis-stream",
                    "output_stream_name": "my-output-kinesis-stream"
                }
            ),
        )

Or, perhaps a single dict:

            property_groups=kaf.PropertyGroups(
                {
                    "FlinkApplicationProperties": {
                        "input_stream_name": "my-input-kinesis-stream",
                        "output_stream_name": "my-output-kinesis-stream",
                    }
                }
            ),

What did you expect to happen?

Able to build PropertyGroups to pass to the Flink application.

What actually happened?

Using keyword args gives the error: TypeError: __init__() got an unexpected keyword argument 'FlinkApplicationProperties'

Using a single dict gives the error: TypeError: __init__() takes 1 positional argument but 2 were given

Environment

• CDK CLI Version : 1.94.1
• Framework Version:
• Node.js Version: v14.15.4
• OS : MacOS
• Language (Version): Python 3.8.6

Other

Looking at the code for PropertyGroups, the constructor signature would seem to be unable to support any arguments at all:

class PropertyGroups:
    def __init__(self) -> None:
        '''(experimental) Interface for building AWS::KinesisAnalyticsV2::Application PropertyGroup configuration.

        :stability: experimental
        '''
        self._values: typing.Dict[str, typing.Any] = {}

Similarly, the Java PropertyGroup.Builder seems to be "empty", having no methods other than build().

I wonder if things are only really working for TypeScript (the language of the example).

---

This is :bug: Bug Report

[iliapolo]

@rgustafson-ie thanks for reporting this.

Looks like the problem is with the jsii assembly, that creates an empty data type for interfaces with dynamic property names.

export interface PropertyGroups {
  readonly [propertyId: string]: {[mapKey: string]: string};
}

"@aws-cdk/aws-kinesisanalytics-flink.PropertyGroups": {
  "assembly": "@aws-cdk/aws-kinesisanalytics-flink",
  "datatype": true,
  "docs": {
    "stability": "experimental",
    "summary": "Interface for building AWS::KinesisAnalyticsV2::Application PropertyGroup configuration."
  },
  "fqn": "@aws-cdk/aws-kinesisanalytics-flink.PropertyGroups",
  "kind": "interface",
  "locationInModule": {
    "filename": "lib/types.ts",
    "line": 39
  },
  "name": "PropertyGroups"
},

cc @RomainMuller

[rgustafson-ie]

For anyone wanting to make this work in the meantime, you can drop down to the L1 CFN.

For example:

        application = kaf.Application(
            self,
            "FlinkApplication",
            code=kaf.ApplicationCode.from_asset(config.asset_path),
            runtime=kaf.Runtime.FLINK_1_11,
            application_name=f"{construct_id}BeamApplication",
            auto_scaling_enabled=True,
            checkpointing_enabled=True,
            snapshots_enabled=False,
        )

        env_property_groups = [
            {
                "PropertyGroupId": "myPropGroupId",
                "PropertyMap": {ENV_SOME_NAME:  "Some value"},
            }
        ]
        kinesis_cfn_application = application.node.default_child
        kinesis_cfn_application.add_property_override(
            "ApplicationConfiguration.EnvironmentProperties.PropertyGroups", env_property_groups
        )

[Jacco]

I experimented a bit with this. I changed the definition around a bit of ApplicationProps in application.ts in aws-kinesisanalytics-flink:

/**
 * Props for creating an Application construct.
 */
export interface ApplicationProps {

 // ... cut away some code

  /**
   * Configuration PropertyGroups. You can use these property groups to pass
   * arbitrary runtime configuration values to your Flink app.
   *
   * @default No property group configuration provided to the Flink app
   */
  readonly propertyGroups?: { [key: string]: { [mapKey: string]: string } };

 // ... cut away some code
}

Then after removing .jsii.tabl.json I reran:

npm run rosetta:extract

The resulting python code:

# bucket: s3.Bucket

flink_app = flink.Application(self, "Application",
    property_groups={
        "FlinkApplicationProperties": {
            "input_stream_name": "my-input-kinesis-stream",
            "output_stream_name": "my-output-kinesis-stream"
        }
    },
    # ...
    runtime=flink.Runtime.FLINK_1_13,
    code=flink.ApplicationCode.from_bucket(bucket, "my-app.jar")
)

I think it has to do with rosetta misinterpreting:

/**
 * Interface for building AWS::KinesisAnalyticsV2::Application PropertyGroup
 * configuration.
 */
export interface PropertyGroups {
  readonly [propertyId: string]: { [mapKey: string]: string };
}

[TheRealAmazonKendra]

Moving this issue to jsii.

[otaviomacedo]

jsii no longer supports index signatures, which means that types like

export interface PropertyGroups {
  readonly [propertyId: string]: { [mapKey: string]: string };
}

will cause lead to a compilation error, unless annotated with @jsii ignore.

To keep backward compatibility, all such types in the CDK have been annotated this way and the properties that had such types were adapted accordingly. In this particular case, as was suggested above, the type became:

interface ApplicationProps {
  ...
  readonly propertyGroups?: { readonly [propertyId: string]: { [mapKey: string]: string } };
  ...
}

[github-actions[bot]]

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.