• Terraform module to deploy lambda that sends notifications about AWS CloudTrail events to Slack
  • Why this module?
  • Example message
  • Delivery delays
• Examples
  • Module deployment with the default ruleset
  • Separating notifications to different Slack channels
    • Module deployment with the default ruleset and different slack channels for different accounts
  • Tracking certain event types
  • User defined rules to match events
    • Module deployment with user-defined rules, list of events to track, and default rule sets
    • Catch SSM Session events for the "111111111" account
  • Ignore rules.
    • Ignore events from the account "111111111".
• About rules and how they are applied
  • Default rules
  • Ignore rules
• Terraform specs
  • Requirements
  • Providers
  • Modules
  • Resources
  • Inputs
  • Outputs
  • License

Terraform module to deploy lambda that sends notifications about AWS CloudTrail events to Slack

Why this module?

This module allows you to get notifications about:

• actions performed by root account (According to AWS best practices, you should use root account as little as possible and use SSO or IAM users)
• API calls that failed due to lack of permissions to do so (could be an indication of compromise or misconfiguration of your services/applications)
• console logins without MFA (Always use MFA for you IAM users or SSO)
• track a list of events that you might consider sensitive. Think IAM changes, network changes, data storage (S3, DBs) access changes. Though we recommend keeping that to a minimum to avoid alert fatigue
• define sophisticated rules to track user-defined conditions that are not covered by default rules (see examples below)
• send notifications to different Slack channels based on event account id

Example message

Delivery delays

The current implementation built upon parsing of S3 notifications, and thus you should expect a 5 to 10 min lag between action and event notification in Slack. If you do not get a notification at all - check CloudWatch logs for the lambda to see if there is any issue with provided filters.

Examples

Module deployment with the default ruleset

# we recomend storing hook url in SSM Parameter store and not commit it to the repo
data "aws_ssm_parameter" "hook" {
  name = "/cloudtrail-to-slack/hook"
}

module "cloudtrail_to_slack" {
  source                         = "fivexl/cloudtrail-to-slack/aws"
  version                        = "2.0.0"
  default_slack_hook_url         = data.aws_ssm_parameter.hook.value
  cloudtrail_logs_s3_bucket_name = aws_s3_bucket.cloudtrail.id
}

resource "aws_cloudtrail" "main" {
  name           = "main"
  s3_bucket_name = aws_s3_bucket.cloudtrail.id
  ...
}

resource "aws_s3_bucket" "cloudtrail" {
  ....
}

Separating notifications to different Slack channels

Module deployment with the default ruleset and different slack channels for different accounts

# we recomend storing hook url in SSM Parameter store and not commit it to the repo
data "aws_ssm_parameter" "default_hook" {
  name = "/cloudtrail-to-slack/default_hook"
}

data "aws_ssm_parameter" "dev_hook" {
  name = "/cloudtrail-to-slack/dev_hook"
}

data "aws_ssm_parameter" "prod_hook" {
  name = "/cloudtrail-to-slack/prod_hook"
}

module "cloudtrail_to_slack" {
  source                         = "fivexl/cloudtrail-to-slack/aws"
  version                        = "2.0.0"
  default_slack_hook_url         = data.aws_ssm_parameter.default_hook.value

  configuration = [
    {
      "accounts": ["123456789"],
      "slack_hook_url": data.aws_ssm_parameter.dev_hook.value
    },
    {
      "accounts": ["987654321"],
      "slack_hook_url": data.aws_ssm_parameter.prod_hook.value
    }
  ]

  cloudtrail_logs_s3_bucket_name = aws_s3_bucket.cloudtrail.id
}

resource "aws_cloudtrail" "main" {
  name           = "main"
  s3_bucket_name = aws_s3_bucket.cloudtrail.id
  ...
}

resource "aws_s3_bucket" "cloudtrail" {
  ....
}

Tracking certain event types

Module deployment with the list of events to track and default rule sets

# we recomend storing hook url in SSM Parameter store and not commit it to the repo
data "aws_ssm_parameter" "hook" {
  name = "/cloudtrail-to-slack/hook"
}

locals {
  # CloudTrail events
  cloudtrail = "DeleteTrail,StopLogging,UpdateTrail"
  # EC2 Instance connect and EC2 events
  ec2 = "SendSSHPublicKey"
  # Config
  config = "DeleteConfigRule,DeleteConfigurationRecorder,DeleteDeliveryChannel,DeleteEvaluationResults"
  # All events
  events_to_track = "${local.cloudtrail},${local.ec2},${local.config}"
}

module "cloudtrail_to_slack" {
  source                         = "fivexl/cloudtrail-to-slack/aws"
  version                        = "2.0.0"
  default_slack_hook_url         = data.aws_ssm_parameter.hook.value
  cloudtrail_logs_s3_bucket_name = aws_s3_bucket.cloudtrail.id
  events_to_track                = local.events_to_track
}

resource "aws_cloudtrail" "main" {
  name           = "main"
  s3_bucket_name = aws_s3_bucket.cloudtrail.id
  ...
}

resource "aws_s3_bucket" "cloudtrail" {
  ....
}

User defined rules to match events

Module deployment with user-defined rules, list of events to track, and default rule sets

# we recomend storing hook url in SSM Parameter store and not commit it to the repo
data "aws_ssm_parameter" "hook" {
  name = "/cloudtrail-to-slack/hook"
}

module "cloudtrail_to_slack" {
  source                         = "fivexl/cloudtrail-to-slack/aws"
  version                        = "2.0.0"
  default_slack_hook_url         = data.aws_ssm_parameter.hook.value
  cloudtrail_logs_s3_bucket_name = aws_s3_bucket.cloudtrail.id
  rules                          = "'errorCode' in event and event['errorCode'] == 'UnauthorizedOperation','userIdentity.type' in event and event['userIdentity.type'] == 'Root'"
  events_to_track                = "CreateUser,StartInstances"
}

Catch SSM Session events for the "111111111" account

# Important! User defined rules should not contain comas since they are passed to lambda as coma separated string
locals {
  cloudtrail_rules = [
      "'userIdentity.accountId' in event and event['userIdentity.accountId'] == '11111111111' and event['eventSource'] == 'ssm.amazonaws.com' and event['eventName'].endswith(('Session'))",
    ]
}

# we recomend storing hook url in SSM Parameter store and not commit it to the repo
data "aws_ssm_parameter" "hook" {
  name = "/cloudtrail-to-slack/hook"
}

module "cloudtrail_to_slack" {
  source                         = "fivexl/cloudtrail-to-slack/aws"
  version                        = "2.0.0"
  default_slack_hook_url         = data.aws_ssm_parameter.hook.value
  cloudtrail_logs_s3_bucket_name = aws_s3_bucket.cloudtrail.id
  rules                          = join(",", local.cloudtrail_rules)
}

Using a custom separator for complex rules containing commas

locals {
  cloudtrail_rules = [
      ...
    ]
  custom_separator = "%"
}

module "cloudtrail_to_slack" {
      ...
  rules           = join(local.custom_separator, local.cloudtrail_rules)
  rules_separator = local.custom_separator
}

Ignore rules.

Ignore events from the account "111111111".

Note! We do recomend fixing alerts instead of ignoring them. But if there is no way you can fix it then there is a way to suppress events by providing ignore rules

# Important! User defined rules should not contain comas since they are passed to lambda as coma separated string
locals {
  cloudtrail_ignore_rules = [
      "'userIdentity.accountId' in event and event['userIdentity.accountId'] == '11111111111'",
    ]
}

# we recomend storing hook url in SSM Parameter store and not commit it to the repo
data "aws_ssm_parameter" "hook" {
  name = "/cloudtrail-to-slack/hook"
}

module "cloudtrail_to_slack" {
  source                         = "fivexl/cloudtrail-to-slack/aws"
  version                        = "2.3.0"
  default_slack_hook_url         = data.aws_ssm_parameter.hook.value
  cloudtrail_logs_s3_bucket_name = aws_s3_bucket.cloudtrail.id
  ignore_rules                   = join(",", local.cloudtrail_ignore_rules)
}

About rules and how they are applied

This module comes with a set of predefined rules (default rules) that users can take advantage of. Rules are python strings that are evaluated in the runtime and should return the bool value. CloudTrail event (see format here) is flattened before processing and should be referenced as event variable So, for instance, to access ARN from the event below, you should use the notation userIdentity.arn

{
  "eventVersion": "1.05",
  "userIdentity": {
    "type": "IAMUser",
    "principalId": "XXXXXXXXXXX",
    "arn": "arn:aws:iam::XXXXXXXXXXX:user/xxxxxxxx",
    "accountId": "XXXXXXXXXXX",
    "userName": "xxxxxxxx"
  },
  "eventTime": "2019-07-03T16:14:51Z",
  "eventSource": "signin.amazonaws.com",
  "eventName": "ConsoleLogin",
  "awsRegion": "us-east-1",
  "sourceIPAddress": "83.41.208.104",
  "userAgent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0",
  "requestParameters": null,
  "responseElements": {
    "ConsoleLogin": "Success"
  },
  "additionalEventData": {
    "LoginTo": "https://console.aws.amazon.com/ec2/v2/home?XXXXXXXXXXX",
    "MobileVersion": "No",
    "MFAUsed": "No"
  },
  "eventID": "0e4d136e-25d4-4d92-b2b2-8a9fe1e3f1af",
  "eventType": "AwsConsoleSignIn",
  "recipientAccountId": "XXXXXXXXXXX"
}

Default rules

# Notify if someone logged in without MFA but skip notification for SSO logins
default_rules.append('event["eventName"] == "ConsoleLogin" ' +
                     'and event["additionalEventData.MFAUsed"] != "Yes" ' +
                     'and "assumed-role/AWSReservedSSO" not in event.get("userIdentity.arn", "")')

# Notify if someone is trying to do something they not supposed to be doing but do not notify
# about not logged in actions since there are a lot of scans for open buckets that generate noise
# This is useful to discover any misconfigurations in your account. Time to time services will try
# to do something but fail due to IAM permissions and those errors are very hard to find using
# other means
default_rules.append('event.get("errorCode", "") == "UnauthorizedOperation"')
default_rules.append('event.get("errorCode", "") == "AccessDenied" ' +
                     'and (event.get("userIdentity.accountId", "") != "ANONYMOUS_PRINCIPAL")')

# Notify about all non-read actions done by root
default_rules.append('event.get("userIdentity.type", "") == "Root" ' +
                     'and not event["eventName"].startswith(("Get", "List", "Describe", "Head"))')

Ignore rules

User can also provide ignore rules. Ignore rules have the same syntax as a default and user defined rules mentioned above. But instead of generating message to Slack on match those rules will cause lambda to ignore an event. Ignore rules tested before default and user defined rules which means that if even is ignored by ignore rules it will not be tested with any other rules.

Terraform specs

Requirements

Providers

Modules

Resources

Inputs

list(object({
    accounts       = list(string)
    slack_hook_url = string
  }))

Outputs

License

Apache 2 Licensed. See LICENSE for full details.