Python class (Interface) is generated out of order

[automartin5000]

Describe the bug

I have a CDK construct library which has been working fine. But suddenly, I think possibly with the addition of named exports, I'm getting an error when attempting to import a construct when it attempts to reference one of the interfaces in a Python CDK app. Looking at the generated Python code, I noticed that the reference occurs before the class is defined.

Here's an example of how the interfaces are defined. Note: These are not really empty.

import { StackProps } from "aws-cdk-lib";

export interface BaseProps {}

export interface SomeAdditionalProps extends SomeProps {} //This may not be relevant

export interface AllTheProps extends StackProps, BaseProps {}

And here's the generated code:

@jsii.data_type(
    jsii_type="@scope/package/moduleName.SomeAdditionalProps",
    jsii_struct_bases=[BaseProps],
    name_mapping={}
)

class SomeAdditionalProps(BaseProps):
    def __init__()....

## A couple hundred lines later
class BaseProps:

Stack trace:

line 133, in <module>
    jsii_struct_bases=[BaseProps],
                       ^^^^^^^^^^^^^^^
NameError: name 'BaseProps' is not defined. Did you mean: ....

Expected Behavior

CDK Synth correctly

Current Behavior

Synth error (See above)

Reproduction Steps

See above code

Possible Solution

No response

Additional Information/Context

No response

SDK version used

5.3.12

Environment details (OS name and version, etc.)

Mac OS Sonoma

[automartin5000]

So based on my testing, what seems to be happening is JSII is generating classes in alphabetical order (is that correct?). But if Class A depends on Class B, then Class B is not yet defined and it throws an error. So I was able to work around this problem by renaming Class B to something like Class AppB so it went before the original Class A.

Does this seem like expected behavior? If so, I would expect that some dependency resolution is needed.

[automartin5000]

I just ran into this again today. I can't possibly be the only person experiencing this?

[otaviomacedo]

Hi, @automartin5000. I can't reproduce the bug based on the information you provided. Is there a complete example you can share?

JSII is generating classes in alphabetical order

jsii generates classes in topological order, where class A is a predecessor of class B if A depends on B in some way (being a subclass is one example). This is done precisely to avoid this issue. The root cause must be something else.

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[automartin5000]

Thanks for the response @otaviomacedo. I'll try to recreate it this week or next

[automartin5000]

Sorry for the delay, here's a clean example:

constructs.ts

import { Construct } from "constructs";
import { AProps } from "./interfaces";

export class TestConstruct extends Construct {
  constructor(scope: Construct, id: string, props: AProps) {
    console.log(`Initialized prop: ${props.testProp}`);
    super(scope, id);
  }
}

interfaces.ts

export interface AProps extends BProps {}

export interface BProps {
  readonly testProp: string;
}

Python app code:

from <construct_library>.test_construct import AProps
from aws_cdk import App

class TestConstruct:
    def __init__(self, scope, id, props: AProps):
        super().__init__(scope, id, props)

test = TestConstruct(App(), 'TestConstruct', AProps(test_prop='test'))

Error:

_init__.py", line 20, in <module>
    jsii_struct_bases=[BProps],
                       ^^^^^^
NameError: name 'BProps' is not defined

Full JSII generated code (anonymized):

import abc
import builtins
import datetime
import enum
import typing

import jsii
import publication
import typing_extensions

from typeguard import check_type

from .._jsii import *

import constructs as _constructs_77d1e7e8


@jsii.data_type(
    jsii_type="<construct_library>.testConstruct.AProps",
    jsii_struct_bases=[BProps],
    name_mapping={"test_prop": "testProp"},
)
class AProps(BProps):
    def __init__(self, *, test_prop: builtins.str) -> None:
        '''
        :param test_prop: 
        '''
        if __debug__:
            type_hints = typing.get_type_hints(_typecheckingstub__6c50750a3eed61a5e3bf6110860acdf626b4cbe1cbe7b2cee25f235971070407)
            check_type(argname="argument test_prop", value=test_prop, expected_type=type_hints["test_prop"])
        self._values: typing.Dict[builtins.str, typing.Any] = {
            "test_prop": test_prop,
        }

    @builtins.property
    def test_prop(self) -> builtins.str:
        result = self._values.get("test_prop")
        assert result is not None, "Required property 'test_prop' is missing"
        return typing.cast(builtins.str, result)

    def __eq__(self, rhs: typing.Any) -> builtins.bool:
        return isinstance(rhs, self.__class__) and rhs._values == self._values

    def __ne__(self, rhs: typing.Any) -> builtins.bool:
        return not (rhs == self)

    def __repr__(self) -> str:
        return "AProps(%s)" % ", ".join(
            k + "=" + repr(v) for k, v in self._values.items()
        )


@jsii.data_type(
    jsii_type="<construct_library>.testConstruct.BProps",
    jsii_struct_bases=[],
    name_mapping={"test_prop": "testProp"},
)
class BProps:
    def __init__(self, *, test_prop: builtins.str) -> None:
        '''
        :param test_prop: 
        '''
        if __debug__:
            type_hints = typing.get_type_hints(_typecheckingstub__a410f745cf17b952ca7c740989abf5daa91608073357db663df598089ac271c0)
            check_type(argname="argument test_prop", value=test_prop, expected_type=type_hints["test_prop"])
        self._values: typing.Dict[builtins.str, typing.Any] = {
            "test_prop": test_prop,
        }

    @builtins.property
    def test_prop(self) -> builtins.str:
        result = self._values.get("test_prop")
        assert result is not None, "Required property 'test_prop' is missing"
        return typing.cast(builtins.str, result)

    def __eq__(self, rhs: typing.Any) -> builtins.bool:
        return isinstance(rhs, self.__class__) and rhs._values == self._values

    def __ne__(self, rhs: typing.Any) -> builtins.bool:
        return not (rhs == self)

    def __repr__(self) -> str:
        return "BProps(%s)" % ", ".join(
            k + "=" + repr(v) for k, v in self._values.items()
        )


class TestConstruct(
    _constructs_77d1e7e8.Construct,
    metaclass=jsii.JSIIMeta,
    jsii_type="<construct_library>.testConstruct.TestConstruct",
):
    def __init__(
        self,
        scope: _constructs_77d1e7e8.Construct,
        id: builtins.str,
        *,
        test_prop: builtins.str,
    ) -> None:
        '''
        :param scope: -
        :param id: -
        :param test_prop: 
        '''
        if __debug__:
            type_hints = typing.get_type_hints(_typecheckingstub__6b4fdcd0c8c3aad835d0884ffd49ae763492010eba1e1acc2290e07ff8485088)
            check_type(argname="argument scope", value=scope, expected_type=type_hints["scope"])
            check_type(argname="argument id", value=id, expected_type=type_hints["id"])
        props = AProps(test_prop=test_prop)

        jsii.create(self.__class__, self, [scope, id, props])


__all__ = [
    "AProps",
    "BProps",
    "TestConstruct",
]

publication.publish()

def _typecheckingstub__6c50750a3eed61a5e3bf6110860acdf626b4cbe1cbe7b2cee25f235971070407(
    *,
    test_prop: builtins.str,
) -> None:
    """Type checking stubs"""
    pass

def _typecheckingstub__a410f745cf17b952ca7c740989abf5daa91608073357db663df598089ac271c0(
    *,
    test_prop: builtins.str,
) -> None:
    """Type checking stubs"""
    pass

def _typecheckingstub__6b4fdcd0c8c3aad835d0884ffd49ae763492010eba1e1acc2290e07ff8485088(
    scope: _constructs_77d1e7e8.Construct,
    id: builtins.str,
    *,
    test_prop: builtins.str,
) -> None:
    """Type checking stubs"""
    pass

[sumupitchayan]

@automartin5000 I think something might be wrong with your Python app code here - why are you defining the TestConstruct class in the Python class, when you already declared the class in constructs.ts above? I'm trying to reproduce this error once again

[automartin5000]

@automartin5000 I think something might be wrong with your Python app code here - why are you defining the TestConstruct class in the Python class, when you already declared the class in constructs.ts above? I'm trying to reproduce this error once again

I'm not defining it in the Python code. The Python code is the generated code from JSII

[sumupitchayan]

Python app code:

from <construct_library>.test_construct import AProps
from aws_cdk import App

class TestConstruct:
    def __init__(self, scope, id, props: AProps):
        super().__init__(scope, id, props)

test = TestConstruct(App(), 'TestConstruct', AProps(test_prop='test'))

Error:

_init__.py", line 20, in <module>
    jsii_struct_bases=[BProps],
                       ^^^^^^
NameError: name 'BProps' is not defined

@automartin5000 this is the code I was referring to. Can you show me the CDK app code you are writing? Also, when/where are you seeing this error?

[automartin5000]

Python app code:

from <construct_library>.test_construct import AProps
from aws_cdk import App

class TestConstruct:
    def __init__(self, scope, id, props: AProps):
        super().__init__(scope, id, props)

test = TestConstruct(App(), 'TestConstruct', AProps(test_prop='test'))

Error:

_init__.py", line 20, in <module>
    jsii_struct_bases=[BProps],
                       ^^^^^^
NameError: name 'BProps' is not defined

@automartin5000 this is the code I was referring to. Can you show me the CDK app code you are writing? Also, when/where are you seeing this error?

That is the Python code being generated by JSII. If you continue to scroll above that, you'll see the code that I wrote. The error is happening at synth.

[sumupitchayan]

Full JSII generated code (anonymized):

import abc
import builtins
import datetime
import enum
import typing

import jsii
import publication
import typing_extensions

from typeguard import check_type

from .._jsii import *

import constructs as _constructs_77d1e7e8


@jsii.data_type(
    jsii_type="<construct_library>.testConstruct.AProps",
    jsii_struct_bases=[BProps],
    name_mapping={"test_prop": "testProp"},
)
class AProps(BProps):
    def __init__(self, *, test_prop: builtins.str) -> None:
        '''
        :param test_prop: 
        '''
        if __debug__:
            type_hints = typing.get_type_hints(_typecheckingstub__6c50750a3eed61a5e3bf6110860acdf626b4cbe1cbe7b2cee25f235971070407)
            check_type(argname="argument test_prop", value=test_prop, expected_type=type_hints["test_prop"])
        self._values: typing.Dict[builtins.str, typing.Any] = {
            "test_prop": test_prop,
        }

    @builtins.property
    def test_prop(self) -> builtins.str:
        result = self._values.get("test_prop")
        assert result is not None, "Required property 'test_prop' is missing"
        return typing.cast(builtins.str, result)

    def __eq__(self, rhs: typing.Any) -> builtins.bool:
        return isinstance(rhs, self.__class__) and rhs._values == self._values

    def __ne__(self, rhs: typing.Any) -> builtins.bool:
        return not (rhs == self)

    def __repr__(self) -> str:
        return "AProps(%s)" % ", ".join(
            k + "=" + repr(v) for k, v in self._values.items()
        )


@jsii.data_type(
    jsii_type="<construct_library>.testConstruct.BProps",
    jsii_struct_bases=[],
    name_mapping={"test_prop": "testProp"},
)
class BProps:
    def __init__(self, *, test_prop: builtins.str) -> None:
        '''
        :param test_prop: 
        '''
        if __debug__:
            type_hints = typing.get_type_hints(_typecheckingstub__a410f745cf17b952ca7c740989abf5daa91608073357db663df598089ac271c0)
            check_type(argname="argument test_prop", value=test_prop, expected_type=type_hints["test_prop"])
        self._values: typing.Dict[builtins.str, typing.Any] = {
            "test_prop": test_prop,
        }

    @builtins.property
    def test_prop(self) -> builtins.str:
        result = self._values.get("test_prop")
        assert result is not None, "Required property 'test_prop' is missing"
        return typing.cast(builtins.str, result)

    def __eq__(self, rhs: typing.Any) -> builtins.bool:
        return isinstance(rhs, self.__class__) and rhs._values == self._values

    def __ne__(self, rhs: typing.Any) -> builtins.bool:
        return not (rhs == self)

    def __repr__(self) -> str:
        return "BProps(%s)" % ", ".join(
            k + "=" + repr(v) for k, v in self._values.items()
        )


class TestConstruct(
    _constructs_77d1e7e8.Construct,
    metaclass=jsii.JSIIMeta,
    jsii_type="<construct_library>.testConstruct.TestConstruct",
):
    def __init__(
        self,
        scope: _constructs_77d1e7e8.Construct,
        id: builtins.str,
        *,
        test_prop: builtins.str,
    ) -> None:
        '''
        :param scope: -
        :param id: -
        :param test_prop: 
        '''
        if __debug__:
            type_hints = typing.get_type_hints(_typecheckingstub__6b4fdcd0c8c3aad835d0884ffd49ae763492010eba1e1acc2290e07ff8485088)
            check_type(argname="argument scope", value=scope, expected_type=type_hints["scope"])
            check_type(argname="argument id", value=id, expected_type=type_hints["id"])
        props = AProps(test_prop=test_prop)

        jsii.create(self.__class__, self, [scope, id, props])


__all__ = [
    "AProps",
    "BProps",
    "TestConstruct",
]

publication.publish()

def _typecheckingstub__6c50750a3eed61a5e3bf6110860acdf626b4cbe1cbe7b2cee25f235971070407(
    *,
    test_prop: builtins.str,
) -> None:
    """Type checking stubs"""
    pass

def _typecheckingstub__a410f745cf17b952ca7c740989abf5daa91608073357db663df598089ac271c0(
    *,
    test_prop: builtins.str,
) -> None:
    """Type checking stubs"""
    pass

def _typecheckingstub__6b4fdcd0c8c3aad835d0884ffd49ae763492010eba1e1acc2290e07ff8485088(
    scope: _constructs_77d1e7e8.Construct,
    id: builtins.str,
    *,
    test_prop: builtins.str,
) -> None:
    """Type checking stubs"""
    pass

@automartin5000 isn't this the Python code generated by JSII^^? I think your Python app code might be incorrect

[automartin5000]

I think your Python app code might be incorrect

What's incorrect about it? I'm importing a class generated by JSII created out of the first two Typescript files

[automartin5000]

@sumupitchayan Just following up to see if you still were unclear on how to replicate?

[automartin5000]

Ok a colleague just pointed out what the confusion might be in my example code. I coincidentally reused the same class name for my Python app as I did for my fake library construct (TestConstruct). But that's irrelevant for the example. To reproduce, you can rename the TestConstruct class (stack) in the Python code and you'll be able to reproduce the bug.

[automartin5000]

I kind of wonder if it's related to this line of code

[JackZheng10]

Experienced the same issue and can confirm that naming the interfaces in alphabetical order based on order of inheritance (where the parent comes first alphabetically) fixed the build issue.

Same issue appeared for classes and same fix applies.

[calebhadley1]

Experienced the same issue

[Macbeth98]

Experienced the same issue with both classes and interfaces, and naming them in alphabetical order fixed the issue!

[mrgrain]

Run into the same issue while working on unrelated stuff. Will look into this.

/**
 * This module demonstrates covariant overrides support in jsii.
 *
 * Covariant overrides allow derived classes to override methods with more specific return types.
 * This was previously not supported because C# didn't allow it, but newer versions of C# (9.0+) do.
 */

/** Base class in the inheritance hierarchy */
export class Superclass {}

/** Derived class that extends Superclass */
export class Subclass extends Superclass {}

/** Further derived class that extends Subclass */
export class SubSubclass extends Subclass {}

export interface IBase {
  readonly something: Superclass;
}

/** Base class with methods and properties that will be overridden with covariant return types */
export class Base implements IBase {
  public readonly something: Superclass = new Superclass();
  public createSomething(): Superclass {
    return new Superclass();
  }
}

/** Middle class in the inheritance chain - doesn't override anything */
export class Middle extends Base {
  public addUnrelatedMember = 3;
}

/**
 * Derived class that demonstrates covariant overrides.
 *
 * Both property and method overrides are covariant and will work in C# 9.0+
 * when the covariant-overrides feature is enabled.
 */
export class Derived extends Middle {
  // This property override is covariant (SubSubclass extends Superclass)
  public readonly something: SubSubclass = new SubSubclass();

  // This method override is covariant and will work in C# 9.0+
  public createSomething(): SubSubclass {
    return new SubSubclass();
  }
}

[mrgrain]

I think I've finally tracked this down. Which is quite something and understandably why it was hard to reproduce.

This seems to happen when the code is in a submodule that uses a pascalCased name as export.

export * as pascalCaseName from './pascal-case-name';

This tracks with your reports, since they all include: testConstruct

When I rename pascalCaseName to pascal or even pascal_case_name, everything works as expected. So somehow the naming is causing the topological sorting of classes inside a module to fail. I'm guessing that the name somehow causes the dependency tracking to fail, thus falling back to alphabetical order.

[github-actions[bot]]

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.

[automartin5000]

Thank you so much @mrgrain! After 2.5 years I thought this issue would never be closed 😭