[Feature Request] environment_name variable option for log_group_name in Logs section

[saranshsingh1]

For our use case, we have the cloudwatch agent sending the logs from EC2 instance managed by AWS Beanstalk.

Apart from the pre-configured standard logs provided by the Beanstalk environment, we want to send other logs as well. We have multiple environments (dev/qa/prod) that send logs to Cloudwatch Logs and the defaults do provide the environment name in the log_group_name which is helpful.

I have tried to replicate something similar for my custom logs before realizing that we can only provide {instance_id}, {hostname}, {local_hostname}, and {ip_address} as variables within the name from the docs.

I did try something like

{
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/va/log/my_custom.log",
            "log_group_name": "{ 'Fn::Sub': '/aws/elasticbeanstalk/${AWSEBEnvironmentName}/var/log/my_custom.log' }",
            "log_stream_name": "{instance_id}"
          },
          ...,
        ]
      }
    }
  }
}

and a few other combinations but it was not fruitful.

It would be very helpful if something like the {environment_name} can also be added as an option which makes it a lot easier to distinguish the different beanstalk environment logs that have the same name as opposed to using the instance_id or something else.

[jhnlsn]

What would the environment_name map back to?

[saranshsingh1]

It would be the name of the environment created by AWS's Beanstalk which is usually made available using the ${AWSEBEnvironmentName} variable when using the cloudformation's Fn::Sub method.

[reyjexter]

Hopefully this feature gets implemented as it's difficult to send custom logs to CloudWatch beginning Amazon Linux 2. Adding environment_name seems needed as you are able to name your CloudWatch logs by given elasticbeanstalk environment name.

On Amazon Linux 1, our custom logs was working fine and stopped sending to CloudWatch after upgrading to Amazon Linux 2 which it appears that you need to modify this file to do that:

/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_beanstalk.json

EDIT:

I made change on some abbreviation which are not correct so from AM2 -> Amazon Linux 2 and AL1 -> Amazon Linux 1

[conorot]

@saranshsingh1 Did you ever find a solution to this?

In my use case I implemented a cloudwatch configuration file in order to start streaming additional metrics like mem. However by creating this manual configuration it completely turns off standard EB logs (access_log, error_log and stdout.log) streaming to Cloudwatch (even though I didn't add a "logs" section to the config). So I'm essentially trying to re-create the default EB logging situation, which logs to a group called /aws/elasticbeanstalk/{environment_name}/var/log/httpd/access_log

That {environment_name} variable doesn't seem available in the cloudwatch agent and using something like {instance_id} would start streaming to a different log group

[conorot]

The only way I've found around this is to use a command to replace a custom variable in the cloudwatch agent file. Here's my full config file which does a few things:

1. sets up metric monitoring for EC2 instances, including things like memory and disk monitoring
2. recreates standard elastic beanstalk log streaming (which is overwritten and turned off if you begin to configure the cloudwatch agent manually)
3. creates a script to replace the environment_name in the cloudwatch agent config file

This is my .ebextensions/01_cloudwatch.config and is functional on the Amazon Linux 2 servers (running apache)

files:  
  "/opt/aws/amazon-cloudwatch-agent/bin/config.json": 
    mode: "000600"
    owner: root
    group: root
    content: |
      {
        "agent": {
          "metrics_collection_interval": 60
        },
        "metrics": {
          "namespace": "System/Linux",
          "append_dimensions": {
            "InstanceId": "${aws:InstanceId}"
          },
          "metrics_collected": {
            "cpu": {
              "measurement": [
                "usage_system"
              ]
            },
            "mem": {
              "measurement": [
                "total",
                "available",
                "available_percent",
                "used",
                "used_percent",
                "free"
              ]
            },
            "netstat": {
              "measurement": [
                "tcp_established",
                "tcp_listen"
              ]
            },
            "processes": {
              "measurement": [
                "blocked",
                "running"
              ]
            },
            "disk": {
              "measurement": [
                "total",
                "free",
                "used",
                "used_percent"
              ]
            }
          }
        },
        "logs": {
          "logs_collected": {
            "files": {
              "collect_list": [
                {
                  "file_path": "/var/log/httpd/access_log",
                  "log_group_name": "/aws/elasticbeanstalk/ENVIRONMENT_NAME/var/log/httpd/access_log"
                },
                {
                  "file_path": "/var/log/httpd/error_log",
                  "log_group_name": "/aws/elasticbeanstalk/ENVIRONMENT_NAME/var/log/httpd/error_log"
                },
                {
                  "file_path": "/var/log/web.stdout.log",
                  "log_group_name": "/aws/elasticbeanstalk/ENVIRONMENT_NAME/var/log/web.stdout.log"
                },
                {
                  "file_path": "/var/log/eb-activity.log",
                  "log_group_name": "/aws/elasticbeanstalk/ENVIRONMENT_NAME/var/log/eb-activity.log"
                },
                {
                  "file_path": "/var/log/eb-engine.log",
                  "log_group_name": "/aws/elasticbeanstalk/ENVIRONMENT_NAME/var/log/eb-engine.log"
                },
                {
                  "file_path": "/var/log/eb-hooks.log",
                  "log_group_name": "/aws/elasticbeanstalk/ENVIRONMENT_NAME/var/log/eb-hooks.log"
                }
              ]
            }
          },
          "log_stream_name": "{instance_id}",
          "force_flush_interval" : 15
        }
      }
container_commands:
  replace_environment_name:
    command: |
      #!/bin/bash
      CURRENT_ENVIRONMENT=$(/opt/elasticbeanstalk/bin/get-config container -k environment_name)
      sed -i "s/ENVIRONMENT_NAME/$CURRENT_ENVIRONMENT/" /opt/aws/amazon-cloudwatch-agent/bin/config.json
  start_cloudwatch_agent: 
    command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json

[dmitriyklyuzov]

@conorot are you running Amazon Linux 1 or 2? Does adding the config.json using .ebextensions work for you? For some reason my file never gets created this way. Instead, I had to use the new Amazon Linux 2 -specific approach of using a platform hook. Overall, my approach is very similar:

• a bash script in .platform/hooks/postdeploy adds the json configuration file into /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/magento2.json
• a deploy script that builds EB config per EB environment and per EB app replaces the value in magento2.json with the actual environment name
• that same script restarts the cloudwatch agent.

But yes, it would be way easier and cleaner to just have the environment_name value available in the script.

[conorot]

@dmitriyklyuzov I'm running Amazon Linux 2. I had a sample of this running using the platform hook so it is possible to do it that way as you say. However I wanted to keep my cloudwatchagent config in a single location for ease of maintenance so I moved it back to this one .ebextensions config file - no particular reason why, I'm just a bit more familiar with the ebextensions approach