aws-otel-java-instrumentation
aws-otel-java-instrumentation copied to clipboard
feat: [Java] EMF Exporter Implementation
Description of changes: Java Version of these PRs:
- https://github.com/aws-observability/aws-otel-python-instrumentation/pull/382
- https://github.com/aws-observability/aws-otel-python-instrumentation/pull/409
- https://github.com/aws-observability/aws-otel-python-instrumentation/pull/410
- https://github.com/aws-observability/aws-otel-python-instrumentation/pull/434
This PR introduces the complete CloudWatch EMF (Embedded Metric Format) exporter implementation for sending OpenTelemetry metrics directly to CloudWatch without requiring a Collector or Agent.
In order to enable this exporter, users MUST set the following environment variables:
OTEL_METRICS_EXPORTER=awsemf-OTEL_EXPORTER_OTLP_LOGS_HEADERS=x-aws-log-group=<log-group-name>,x-aws-log-stream=<log-stream-name>, x-aws-metric-namespace=<namespace>AWS_REGION=<region>ORAWS_DEFAULT_REGION=<region>
This PR includes:
- EMF MetricRecord translation for for unified representation of all OTel metric types with log creation with unit mapping from OpenTelemetry to CloudWatch-compatible units
- Automatic log group and stream creation with retry logic
- Supported CloudWatch Logs integration with batching and constraint handling (256KB event limit, 1MB request limit, timestamp limits)
- Support for metric grouping by attributes and timestamps for EMF log generation
- Support for Gauge, Sum, Histogram, and ExponentialHistogram metric types
TODO:
- On the next PR, will integrate the Console EMF exporter into AWS Lambda environments to validate EMF log formatting and ensure consistent behavior in Lambda runtime: https://github.com/aws-observability/aws-otel-python-instrumentation/pull/437
Testing:
-
Added unit tests to validate EMF exporter configuration scenarios, including parameterized tests for both valid configurations (supporting AWS_REGION and AWS_DEFAULT_REGION) and invalid configurations (missing headers, wrong exporter type, missing region). The tests ensure the EMF exporter is correctly enabled only when all required environment variables are properly configured.
-
Manual end to end testing with the following environment variables to ensure the EMF logs show up:
AWS_REGION=us-east-1OTEL_METRICS_EXPORTER=awsemfOTEL_EXPORTER_OTLP_LOGS_HEADERS=x-aws-log-group=test,x-aws-log-stream=default,x-aws-metric-namespace=testNamespaceOTEL_RESOURCE_ATTRIBUTES=service.name=testService,aws.log.group.names=test,cloud.resource_id=agent-12345OTEL_LOGS_EXPORTER=noneOTEL_TRACES_EXPORTER=none
Example EMF log emitted:
{
"otel.resource.process.command_args": "[/Library/Java/JavaVirtualMachines/amazon-corretto-21.jdk/Contents/Home/bin/java, -javaagent:/Users/liustve/aws-otel-java-instrumentation/otelagent/build/libs/aws-opentelemetry-agent-2.18.0-SNAPSHOT.jar, -Dfile.encoding=UTF-8, -Dsun.stdout.encoding=UTF-8, -Dsun.stderr.encoding=UTF-8, -jar, /Users/liustve/aws-otel-java-instrumentation/sample-apps/springboot/build/libs/springboot-2.11.0-SNAPSHOT.jar]",
"otel.resource.host.arch": "aarch64",
"otel.resource.host.name": "7cf34dd812df",
"otel.resource.service.instance.id": "a0399d3c-b856-43ae-b374-fe66dee41ce8",
"otel.resource.aws.log.group.names": "test",
"jvm.class.unloaded": 1,
"otel.resource.service.name": "testSErvice",
"_aws": {
"CloudWatchMetrics": [
{
"Metrics": [
{
"Unit": "Count",
"Name": "jvm.cpu.count"
},
{
"Unit": "Count",
"Name": "jvm.class.loaded"
},
{
"Name": "jvm.cpu.recent_utilization"
},
{
"Unit": "Seconds",
"Name": "jvm.cpu.time"
},
{
"Unit": "Count",
"Name": "jvm.class.count"
},
{
"Unit": "Count",
"Name": "jvm.class.unloaded"
}
],
"Namespace": "testNamespace"
}
],
"Timestamp": 1758761023497
},
"otel.resource.cloud.resource_id": "agent-12345",
"jvm.class.count": 14264,
"Version": "1",
"otel.resource.process.pid": "46822",
"otel.resource.os.description": "Mac OS X 15.6.1",
"otel.resource.telemetry.distro.name": "opentelemetry-java-instrumentation",
"otel.resource.os.type": "darwin",
"otel.resource.telemetry.sdk.name": "opentelemetry",
"otel.resource.telemetry.distro.version": "2.18.0-aws-SNAPSHOT",
"otel.resource.process.runtime.description": "Amazon.com Inc. OpenJDK 64-Bit Server VM 21.0.8+9-LTS",
"otel.resource.process.runtime.version": "21.0.8+9-LTS",
"jvm.cpu.recent_utilization": 0,
"otel.resource.process.executable.path": "/Library/Java/JavaVirtualMachines/amazon-corretto-21.jdk/Contents/Home/bin/java",
"otel.resource.telemetry.sdk.version": "1.52.0",
"jvm.cpu.count": 12,
"jvm.class.loaded": 14265,
"otel.resource.process.runtime.name": "OpenJDK Runtime Environment",
"otel.resource.telemetry.sdk.language": "java",
"jvm.cpu.time": 7.756965
}
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.