cdktf: Synth ignore code changes and keep with old values, also incomplete

[djakielski]

trafficstars

Expected Behavior

I had an Lambda function from a generated module (https://github.com/cloudposse/terraform-aws-lambda-function) and change name of handler attribute from mail.handler to index.handler. I also add a Policy attachment. So it should change the handler property and create new policy.

import {Construct} from 'constructs';
import {LambdaFunction} from '../.gen/modules/lambda-function';
import {AssetType, Fn, TerraformAsset} from 'cdktf';
import * as path from 'node:path';
import {DataAwsIamPolicyDocument} from '@cdktf/provider-aws/lib/data-aws-iam-policy-document';
import {IamPolicy} from '@cdktf/provider-aws/lib/iam-policy';

export interface EmailExtractorConfig {
    readonly extractedBucketPath: string;
    readonly inboundBucketPath: string;
    readonly functionArtifactPath: string;
}

export class EmailExtractor extends Construct {
    private readonly lambdaFunction: LambdaFunction;

    constructor(scope: Construct, id: string, config : EmailExtractorConfig) {
        super(scope, id);
        const policy = new IamPolicy(this, `${id}-policy`, {
            name: "email-extractor",
            policy: new DataAwsIamPolicyDocument(this, `${id}-policy-document`, {
                statement: [{
                    effect: 'Allow',
                    actions: ['s3:GetObject'],
                    resources: [`arn:aws:s3:::${Fn.replace(config.inboundBucketPath, "s3://", "")}`],
                },{
                    effect: 'Allow',
                    actions: ['s3:PutObject'],
                    resources: [`arn:aws:s3:::${Fn.replace(config.extractedBucketPath, "s3://", "")}`],
                }]
            }).json
        })
        this.lambdaFunction = new LambdaFunction(this, "function", {
            functionName: 'email-extractor',
            roleName: 'email-extractor',
            runtime: 'nodejs20.x',
            handler: 'index.handler',
            filename: new TerraformAsset(this, 'lambda-asset', {
                type: AssetType.ARCHIVE,
                path: path.resolve(config.functionArtifactPath),
            }).path,
            ephemeralStorageSize: 1024,
            memorySize: 1024,
            publish: true,
            tracingConfigMode: 'Active',
            lambdaEnvironment: {
                variables: {
                    INBOUND_BUCKET_PATH: config.inboundBucketPath,
                    EXTRACTED_BUCKET_PATH: config.extractedBucketPath,
                }
            },
            customIamPolicyArns: [policy.arn]
        })

    }

    getLambdaArn() : string {
        return this.lambdaFunction.arnOutput;
    }
}

Actual Behavior

No changes detected

When i run cdktf deploy or cdktf synth in cdk.tf.json will sill be the old value "mail.handler". Also when I delete cdktf.out folder. I had no idea where the old value comes from. Maybe from remote backend?

Changes on Environment Variables are detected and will deployed.

...
"module": {
    "order-reader-email-extractor_function_DBF17BD2": {
      "//": {
        "metadata": {
          "path": "order-reader/order-reader-email-extractor/function",
          "uniqueId": "order-reader-email-extractor_function_DBF17BD2"
        }
      },
      "ephemeral_storage_size": 1024,
      "filename": "assets/order-reader-email-extractor_lambda-asset_7DBD78E7/9D1305C6223E9D0C404D6FA27DFDF87F/archive.zip",
      "function_name": "email-extractor",
      "handler": "mail.handler",
      "lambda_environment": {
        "variables": {
          "EXTRACTED_BUCKET_PATH": "s3://${aws_s3_bucket.order-reader-inbound-mail_E8ABD570.bucket}/extracted2",
          "INBOUND_BUCKET_PATH": "inbound-orders"
        }
      },
      "memory_size": 1024,
      "publish": true,
      "role_name": "email-extractor",
      "runtime": "nodejs20.x",
      "source": "cloudposse/lambda-function/aws",
      "tracing_config_mode": "Active",
      "version": "~> 0.6"
    },
...

Steps to Reproduce

1. Deploy Lambda with cdktf
2. Change handler name
3. Deploy again

Versions

language: typescript cdktf-cli: 0.20.8 node: v20.15.1 cdktf: 0.20.8 constructs: 10.3.0 jsii: null terraform: 1.7.2 arch: arm64 os: darwin 23.6.0 providers sveba/netcupdns@~> 1.2 (LOCAL) terraform provider version: 1.2.0 @cdktf/provider-aws (PREBUILT) terraform provider version: 5.65.0 prebuilt provider version: 19.33.0 cdktf version: ^0.20.0

Providers

┌─────────────────┬──────────────────┬─────────┬────────────┬─────────────────────┬─────────────────┐ │ Provider Name │ Provider Version │ CDKTF │ Constraint │ Package Name │ Package Version │ ├─────────────────┼──────────────────┼─────────┼────────────┼─────────────────────┼─────────────────┤ │ sveba/netcupdns │ 1.2.0 │ │ ~> 1.2 │ │ │ ├─────────────────┼──────────────────┼─────────┼────────────┼─────────────────────┼─────────────────┤ │ aws │ 5.65.0 │ ^0.20.0 │ │ @cdktf/provider-aws │ 19.33.0 │ └─────────────────┴──────────────────┴─────────┴────────────┴─────────────────────┴─────────────────┘

Gist

No response

Possible Solutions

No response

Workarounds

No response

Anything Else?

No response

References

No response

Help Wanted

• [ ] I'm interested in contributing a fix myself

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[djakielski]

I had this issue also with other ressorces. Seams like an common issue.

[djakielski]

I also detect that new resources within a custruct were completly ignored.

I add a dynomoDB table and it is missing in the synth stack result.

export class OrderReaderStepFunction extends Construct {
    private readonly stepFunction: StepFunction;
    private readonly stateDb: DynamodbTable;
    constructor(scope: Construct, id: string, config : OrderReaderStepFunctionConfig) {
        super(scope, id, );
        const file = new TerraformAsset(this, `definition`, {
            path: path.resolve(__dirname, '../step-functions/orderReader.asl.json'),
            type: AssetType.FILE,
            assetHash: Fn.filemd5(path.resolve(__dirname, '../step-functions/orderReader.json'))
        });
        this.stateDb = new DynamodbTable(scope, `state-db`, {
            name: "order-reader",
            hashKey: "jobId",
            attribute: [{
                name: "jobId",
                type: "S"
            }]
        });
        this.stepFunction = new StepFunction(this, "function", {
            name: 'order-reader',
            tracingEnabled: true,
            definition: Fn.templatefile(file.path, {
                emailExtractorArn: config.emailExtractorArn,
                stateDbName: this.stateDb.name,
                topicClassifierArn: config.topicClassifierArn,
            }),
            loggingConfiguration: {
                level: 'ALL',
                include_execution_data: true
            },
            ....

This construct is a part of the stack

class OrderReader extends TerraformStack {
  constructor(scope: Construct, id: string) {
    super(scope, id);
    new AwsProvider(this, "aws");
   ...
   //More construct opbjects
   ...
    new OrderReaderStepFunction(this, `${id}-step-function`,{
      emailExtractorArn: emailExtractor.getLambdaArn(),
      triggerBucket: ses.getInboundS3Bucket(),
      triggerBucketPath: ses.getInboundS3Path(),
      topicClassifierArn: topicClassifier.getLambdaArn()
    });

[DanielMSchmidt]

We had issues like this at some point in the past with Typescript when folks where using tsc instead of ts-node and their build step then did not compile the program again, therefore executing the old js code. Other than that, CDKTF has no access to the old value during synth. I would suggest removing the cdktf.out folder as a test, maybe cdktf failed to produce something new and threw an error?

Other than that I notice this.stateDb = new DynamodbTable(scope, "state-db", { is using scope instead of this, is that by accident?

[djakielski]

Hey @DanielMSchmidt sounds intresting. Did you mean the app command or in package.json? The DynamodbTable was just a test, to check if it makes any difference.

I remove cdktf.out folder for so many times without any results.

cdktf.json

{
  "language": "typescript",
  "app": "npx ts-node main.ts",
  "projectId": "xxxxxxxxxxxx",
  "sendCrashReports": "true",
  "terraformProviders": [
    {
      "name": "netcupdns",
      "source": "sveba/netcupdns",
      "version": "~> 1.2"
    },
    {
      "name": "counters",
      "source" :"RutledgePaulV/counters",
      "version": "~> 0.0.5"
    }
  ],
  "terraformModules": [
    {
      "name": "step-function",
      "source": "cloudposse/step-functions/aws",
      "version": "~> 0.2"
    },
    {
      "name": "ses-domain",
      "source": "cloudposse/ses/aws",
      "version": "~> 0.25"
    },
    {
      "name": "lambda-function",
      "source": "cloudposse/lambda-function/aws",
      "version": "~> 0.6"
    },
    {
      "name": "event-bridge",
      "source": "terraform-aws-modules/eventbridge/aws",
      "version": "~> 3.10"
    }
  ],
  "context": {
  }
}

package.json

{
  "name": "order-reader",
  "version": "1.0.0",
  "main": "main.js",
  "types": "main.ts",
  "license": "MPL-2.0",
  "private": true,
  "packageManager": "[email protected]",
  "scripts": {
    "postinstall": "npx cdktf get",
    "get": "npx cdktf get",
    "plan": "pnpm run build && cdktf plan",
    "deploy": "pnpm run build && cdktf deploy",
    "build": "turbo run build",
    "synth": "npx cdktf synth",
    "compile": "tsc --pretty",
    "watch": "tsc -w",
    "test": "jest",
    "test:watch": "jest --watch",
    "upgrade": "pnpm i -w cdktf@latest cdktf-cli@latest",
    "upgrade:next": "pnpm i -w cdktf@next cdktf-cli@next"
  },
  "engines": {
    "node": ">=18.0"
  },
  "dependencies": {
    "@cdktf/provider-aws": "19.33.0",
    "asl-types": "^1.2.1",
    "cdktf": "^0.20.9",
    "constructs": "^10.3.0"
  },
  "devDependencies": {
    "cdktf-cli": "^0.20.9",
    "@types/jest": "^29.5.12",
    "@types/node": "^22.5.1",
    "jest": "^29.7.0",
    "ts-jest": "^29.2.5",
    "ts-node": "^10.9.2",
    "turbo": "^2.1.3",
    "typescript": "^5.5.4"
  }
}

[DanielMSchmidt]

I mean whatever gets executed, if you run cdktf synth that would be the app command

[djakielski]

Hey, I found the solution! I wrapped my CDKTF project in a Turbo repo, which also handles the build process for our Lambda functions. When I executed the build job, the Node.js build step within the CDKTF project ran as well. This caused outdated JavaScript files to be generated, and it appears that CDKTF prioritizes these JS files over the original TypeScript files.

The solution was to delete js files and disable build step in turbo.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.