aws
(@aws_cdk) Python constructs do not implement a compatible interface
Describe the bug
Similar (old) issue: https://github.com/aws/aws-cdk/issues/15651
aws_ec2.SubnetSelection cannot be initialized from a list of aws_ec2.Subnet's because aws_ec2.ISubnet is not a compatible interface
Fix
We yanked construct==10.3.1 and released construct==10.3.2 with correct dependency constraints.
Make sure your project is not using construct==10.3.1 and not using typeguard==4.3.0
Workaround
In requirements.txt, pin the version of typeguard:
typeguard==2.13.3
Expected Behavior
Passing a list of aws_ec2.Subnet's to an aws_ec2.SubnetSelection can initialize properly
Current Behavior
Error Message:
Traceback (most recent call last):
File "/home/myuser/testdir/app2/app.py", line 34, in <module>
App2(app, 'App2', env=env)
File "/home/myuser/testdir/.venv/lib/python3.11/site-packages/jsii/_runtime.py", line 118, in __call__
inst = super(JSIIMeta, cast(JSIIMeta, cls)).__call__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/myuser/testdir/app2/app.py", line 27, in __init__
ec2.SubnetSelection(subnets=subnet_construct_list) # this goes boom
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/myuser/testdir/.venv/lib/python3.11/site-packages/aws_cdk/aws_ec2/__init__.py", line 85174, in __init__
check_type(argname="argument subnets", value=subnets, expected_type=type_hints["subnets"])
File "/home/myuser/testdir/.venv/lib/python3.11/site-packages/aws_cdk/aws_ec2/__init__.py", line 2602, in check_type
typeguard.check_type(value=value, expected_type=expected_type, collection_check_strategy=typeguard.CollectionCheckStrategy.ALL_ITEMS) # type:ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/myuser/testdir/.venv/lib/python3.11/site-packages/typeguard/_functions.py", line 106, in check_type
check_type_internal(value, expected_type, memo)
File "/home/myuser/testdir/.venv/lib/python3.11/site-packages/typeguard/_checkers.py", line 861, in check_type_internal
checker(value, origin_type, args, memo)
File "/home/myuser/testdir/.venv/lib/python3.11/site-packages/typeguard/_checkers.py", line 433, in check_union
raise TypeCheckError(f"did not match any element in the union:\n{formatted_errors}")
typeguard.TypeCheckError: list did not match any element in the union:
Sequence[aws_cdk.aws_ec2.ISubnet]: item 0 is not compatible with the ISubnet protocol because it has no method named '__jsii_proxy_class__'
NoneType: is not an instance of NoneTypeTraceback (most recent call last):
Reproduction Steps
requirements.txt
aws-cdk-lib==2.162.0
constructs>=10.0.0,<11.0.0
requirements-dev.txt
pytest==6.2.5
boto3
app1/app.py
import os
from aws_cdk import (
App,
Environment,
Stack,
aws_ec2 as ec2,
aws_ssm as ssm,
)
from constructs import Construct
class App1(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
vpc = ec2.Vpc(
self, 'Vpc',
max_azs=1,
create_internet_gateway=False,
subnet_configuration=[
ec2.SubnetConfiguration(
name='PublicSubnets',
subnet_type=ec2.SubnetType.PUBLIC,
cidr_mask=26,
),
ec2.SubnetConfiguration(
name='PrivateSubnets',
subnet_type=ec2.SubnetType.PRIVATE_WITH_EGRESS,
cidr_mask=20,
),
],
nat_gateways=1,
restrict_default_security_group=True,
)
private_subnets = ','.join([s.subnet_id for s in vpc.private_subnets])
ssm.StringParameter(
self, 'SubnetParam',
parameter_name='/my/subnets',
string_value=private_subnets,
)
env = Environment(account=os.getenv('CDK_DEFAULT_ACCOUNT'),
region=os.getenv('CDK_DEFAULT_REGION'))
app = App()
App1(app, 'App1', env=env)
app.synth()
app2/app.py
import os
import boto3
from aws_cdk import (
App,
Environment,
Stack,
aws_ec2 as ec2,
)
from constructs import Construct
def get_parameter() -> list:
ssm_client = boto3.client('ssm', region_name='us-east-2')
response = ssm_client.get_parameter(Name='/my/subnets')
val = response['Parameter']['Value']
return val.split(',')
class App2(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
subnet_list = get_parameter()
subnet_construct_list = [
ec2.Subnet.from_subnet_id(self, s, subnet_id=s) for s in subnet_list
]
ec2.SubnetSelection(subnets=subnet_construct_list) # this goes boom
env = Environment(account=os.getenv('CDK_DEFAULT_ACCOUNT'),
region=os.getenv('CDK_DEFAULT_REGION'))
app = App()
App2(app, 'App2', env=env)
app.synth()
Deploy:
$ python3 -m venv .venv && source .venv/bin/activate
$ pip install -r requirements.txt -r requirements-dev.txt
$ cdk deploy -vv --require-approval never --app "python3 app1/app.py"
$ cdk deploy -vv --require-approval never --app "python3 app2/app.py"
Observe the error. I also confirmed that a print statement for subnet_list prints ['subnet-0b39XXXXXX']
Possible Solution
Not sure
Additional Information/Context
Package Version
attrs 24.2.0 aws-cdk.asset-awscli-v1 2.2.206 aws-cdk.asset-kubectl-v20 2.1.3 aws-cdk.asset-node-proxy-agent-v6 2.1.0 aws-cdk.cloud-assembly-schema 38.0.1 aws-cdk-lib 2.162.0 boto3 1.35.38 botocore 1.35.38 cattrs 23.2.3 constructs 10.3.1 importlib_resources 6.4.5 iniconfig 2.0.0 jmespath 1.0.1 jsii 1.103.1 packaging 24.1 pip 24.0 pluggy 1.5.0 publication 0.0.3 py 1.11.0 pytest 6.2.5 python-dateutil 2.9.0.post0 s3transfer 0.10.3 setuptools 65.5.0 six 1.16.0 toml 0.10.2 typeguard 4.3.0 typing_extensions 4.12.2 urllib3 2.2.3
SDK version used
CDK 2.162.0 (build c8d7dd3), python 3.11, node 18 & 20
Environment details (OS name and version, etc.)
Amazon Linux 2023 & Ubuntu 20.04 on WSL
Our team has also found a smaller example that doesn't work and produces a much more obscure error. Stack contents:
vpc: aws_ec2.Vpc = aws_ec2.Vpc(
self,
"Vpc",
max_azs=1,
subnet_configuration=[
aws_ec2.SubnetConfiguration(
name="PublicSubnets",
subnet_type=aws_ec2.SubnetType.PUBLIC,
cidr_mask=26,
),
aws_ec2.SubnetConfiguration(
name="PrivateSubnets",
subnet_type=aws_ec2.SubnetType.PRIVATE_WITH_EGRESS,
cidr_mask=20,
),
],
nat_gateways=1,
restrict_default_security_group=True,
)
aws_ec2.SecurityGroup(
self,
"SecurityGroup",
vpc=vpc,
allow_all_outbound=True,
allow_all_ipv6_outbound=False,
description="This is a test",
)
Error:
typeguard.TypeCheckError: aws_cdk.aws_ec2.Vpc is not compatible with the IVpc protocol because its 'add_client_vpn_endpoint' method has mandatory keyword-only arguments in its declaration: cidr, server_certificate_arn
We're running into similar issues with aws_cdk.BootstraplessSynthesizer and aws_cdk.IReusableStackSynthesizer:
typeguard.TypeCheckError: aws_cdk.BootstraplessSynthesizer did not match any element in the union:
aws_cdk.IReusableStackSynthesizer: is not compatible with the IReusableStackSynthesizer protocol because its 'add_docker_image_asset' method has mandatory keyword-only arguments in its declaration: source_hash
NoneType: is not an instance of NoneType
I'm also experiencing this on deploys that were working yesterday before the aws-cdk upgrade. I believe pinning the typeguard version to <3 will fix for now, but would be good to get this resolved.
This seems to be intentional behavior in typeguard >=3
https://github.com/agronholm/typeguard/blob/cf25d56dc0dbf6bb2f51ea29da8436b368ed4857/src/typeguard/_checkers.py#L171-L181
But I don't understand yet why. PEP3102 explicitly allows keyword-only arguments without defaults.
This check seems to have been added ages ago, so the pre-conditions to trigger this check must have changed https://github.com/agronholm/typeguard/blob/cf25d56dc0dbf6bb2f51ea29da8436b368ed4857/docs/versionhistory.rst?plain=1#L518
Check seems to pass with typeguard==3.0.2 if I add @typing.runtime_checkable to the protocol.
Same for typeguard-4.0.0 and typeguard-4.1.0 and typeguard-4.2.1
=> typeguard 4.3.0 is introducing this issue.
Seems to be introduced by this commit: https://github.com/agronholm/typeguard/commit/241d120de7d5e724a9dac10f529318273738a21f#diff-0ac00416d20efb96e963a903c7c8f8e7a5070064af5b4673ad441c7e3114265eR727
We yanked the 10.3.1 release of constructs from PyPI as a quick mitigation for most users. https://pypi.org/project/constructs/10.3.1/
Opened an issue with typeguard to better understand this requirement https://github.com/agronholm/typeguard/issues/495
Another example
from aws_cdk import App, Stack
from aws_cdk import aws_lambda, aws_s3
from aws_cdk import aws_s3_notifications as s3_notify
app = App()
stack = Stack(app, "bucket-stack")
bucket = aws_s3.Bucket(stack, "Bucket")
bucket.add_event_notification(
event=aws_s3.EventType.OBJECT_CREATED,
dest=s3_notify.LambdaDestination(
aws_lambda.Function.from_function_arn(
stack,
"ObjectCreatedLambda",
"arn:aws:lambda:us-west-2:460106496004:function:ObjectCreatedLambda-qQkkxlUaClaJ",
)
),
)
app.synth()
It appears the issue has now been resolved, and I've confirmed this with AWS, who directed me to this solution. We implemented the same workaround yesterday by specifying the Typeguard version. I appreciate everyone who worked on resolving this issue in a timely manner.
notes: Successful deployment: Typeguard was downloaded twice: Downloading typeguard-4.3.0-py3-none-any.whl.metadata (3.7 kB) Downloading typeguard-2.13.3-py3-none-any.whl.metadata (3.6 kB)
Failed deployment: Typeguard was downloaded only once: Downloading typeguard-4.3.0-py3-none-any.whl (35 kB)
I am not sure why typeguard is installed twice every time aws-cdk-lib is ran but it's how all our release look.
I am not sure why typeguard is installed twice every time aws-cdk-lib is ran but it's how all our release look.
This might be a quirk of your package manager. Using poetry I can see typeguard being downloaded only once.
I am not sure why typeguard is installed twice every time aws-cdk-lib is ran but it's how all our release look.
This might be a quirk of your package manager. Using
poetryI can seetypeguardbeing downloaded only once.
Good suggestion but looks like to me how pip handles the install?
Reason I created a fresh virtual environment using python -m venv .venv, then ran a pip install aws-cdk-lib --no-cache-dir. Even then I still get typeguard to appear twice.
Reason I created a fresh virtual environment using
python -m venv .venv,then ran apip install aws-cdk-lib --no-cache-dir. Even then I still get typeguard to appear twice.
Yeah, I see the same. I think pip isn't just very clever in resolving dependencies. aws-cdk-lib currently still incorrectly declares compatibility with typeguard-4.3.0. We are working on a fix for that. Because constructs has been fixed, you will end using a compatible version of typeguard. I guess pip just downloads both before making this choice.
I actually got the dependency via transitivity. I mentioned explicitly stating the version to make it easier to reproduce.
In fact, upon checking the changes to my Poetry lock file, I noticed that a particular commit caused "typeguard" to bump from 2.13 to 4.3. That was due to a cascade of upgrades allowed by the fact that
aws-cdk-asset-kubectl-v20declared support fortypeguard = ">=2.13.3,<5.0.0"
This might be resolved in typeguard 4.4.0 https://github.com/agronholm/typeguard/issues/465
@mrgrain I just tried with 4.4.0 and got the following:
File "/usr/local/lib/python3.12/site-packages/constructs/__init__.py", line 800, in __init__
--
958 | check_type(argname="argument scope", value=scope, expected_type=type_hints["scope"])
959 | TypeError: check_type() got an unexpected keyword argument 'argname'
960 | [22:47:11] failed command: python3 app.py
