Better support for API Gateway execution logs including improved and additional settings

[wgillisaws]

Describe the feature

Enabling execution logs on API Gateway without the full requests/responses being logged now happens through the following 2 properties on the SpecRestApi; deployOptions.loggingLevel: MethodLoggingLevel.INFO and deployOptions.dataTraceEnabled: false. This is somewhat confusing as it isn't clear to the developer that this deals with execution logs. The documentation also does not mention "execution logs" in the section of this property. Changing the retention policy of the log group that is created for these execution logs is not possible through CDK, which is possible for access logs. This is a lacking feature and as Cloudwatch logs can be quiet pricy, it is a feature that would be highly beneficial to many.

It would be more convenient if the documentation at least mentions that this is with regards to execution logs or if the property names that are being used are more clear on this. Having a property such as accessLogDestination but for execution logs (eg. executionLogDestination) would be highly beneficial as it would allow to modify the retention policy on the log group.

Use Case

This feature would be useful as it is confusing at the moment on how to enable execution logs. It would be cost saving as well as it is currently impossible to define a retention period for the log group through CDK.

Proposed Solution

• Update the documentation to include the words "execution logs"
• Change the naming and usage of properties for execution logs
  • there could be a property to turn execution logs on
  • there could be a property to set the log level (INFO | ERROR)
  • there could be a property to set if the full requests/responses need to be logged
• Add a property to pass a custom created log group instead of automatically creating one with the API Gateway deployment

Other Information

No response

Acknowledgements

• [ ] I may be able to implement this feature request
• [X] This feature might incur a breaking change

CDK version used

2.10.0

Environment details (OS name and version, etc.)

MacOS Monterey V12.3

[shellscape]

Just ran into this exact issue. I would love to see support for modifying the execution log group.

References: #15816

[awscomckee]

Ran into this issue today as well, still thinking this would be a beneficial feature-add if not already planned for the api_gatewayv2 construct.

[wtfiwtz]

currently impossible to define a retention period for the log group through CDK

You can now with the latest version... using a log group you pass to deployOptions. Something like this...

  const gatewayLogGroup = new cdk.aws_logs.LogGroup(context, "LogGroup", {
    logGroupName: `/aws/lambda/logGroupName`,
    retention: logs.RetentionDays.ONE_YEAR,
    removalPolicy: cdk.RemovalPolicy.RETAIN_ON_UPDATE_OR_DELETE
  }
  

    deployOptions: {
      stageName: "stageName",
      accessLogDestination: new apigateway.LogGroupLogDestination(gatewayLogGroup),
      accessLogFormat: apigateway.AccessLogFormat.jsonWithStandardFields({
        caller: false,
        httpMethod: true,
        ip: true,
        protocol: true,
        requestTime: true,
        resourcePath: true,
        responseLength: true,
        status: true,
        user: true
      })
    },

[antoniordz96]

@wtfiwtz I think it's always been possible for access logs but I think what the original issue is talking about is execution logs. Log group name would follow API-Gateway-Execution-Logs_{rest-api-id}/{stage_name} so we can't create it ourselves as we don't know the restapi id till after deployment

[awscomckee]

That's correct, @antoniordz96 summarized it.

[antoniordz96]

I don't have bandwidth at the moment to test this but I think one could do the following.

1. provision apigateway
2. update the function below to take in restApiID instead of restApiName and the apigw stageName and optionally the retention period
3. add a dependsOn between the apigw and customResource so you'll have the restApiID

We use the custom resource below in our internal cdk lib to create/update an existing access log group with the retention policy. Never been a fan of doing this via custom resource but it works.

  protected createApiGatewayAccessLogsGroup(restApiName: string, retentionDays: aws_logs.RetentionDays,
    removalPolicy: RemovalPolicy): aws_api.LogGroupLogDestination {

    const logGroupName = this.APIGW_LOG_GROUP_PREFIX + restApiName;

    // This is a custom resource that creates/updates logGroup.
    // This uses an aws lambda under the hood
    // TODO(ccoe) reduce the permissions of this custom resource to avoid nag issues
    const customResource = new aws_logs.LogRetention(this, 'AccessLogsLogGroupCustomResource', {
      logGroupName: logGroupName,
      retention: retentionDays,
      removalPolicy: removalPolicy,
    });

    const logGroup = aws_logs.LogGroup.fromLogGroupArn(this, 'AccessLogsLogGroup', customResource.logGroupArn);

    return new aws_api.LogGroupLogDestination(logGroup);
  }