ECS open telemetry integration and CI enhanced

[jakeskyaws]

trafficstars

Issue: #827

OpenTelemetry Integration and Container Insights Enhancement

Description of changes:

This PR adds OpenTelemetry integration and Container Insights configuration to the ECS deployment, inline with the EKS implementation.

Major Changes

• Added OpenTelemetry instrumentation support via AWS Distro for OpenTelemetry (ADOT)
• Added configurable Container Insights settings
• Added CloudWatch log retention configuration
• Updated documentation to reflect new features

Details

• New variables:

  • opentelemetry_enabled: Toggle for OpenTelemetry integration
  • container_insights_setting: Control Container Insights mode
  • log_group_retention_days: Configure log retention period

• Added ADOT collector configuration for container insights and AWS XRay:

  • https://github.com/aws-observability/aws-otel-collector/blob/main/config/ecs/container-insights/otel-task-metrics-config.yaml

• Enhanced IAM policies to support:

  • ADOT collector permissions
  • CloudWatch logs access
  • X-Ray trace submission

• Updated service configurations to support OpenTelemetry

Documentation

• Updated README with new variables and features
• Added descriptions for observability features

Testing Done

• Verified trace collection with X-Ray
• Confirmed Container Insights functionality
• Validated log group retention settings

Example of tracemap using ADOT

Example of Contain Insights via ADOT

Example of Enhanced Container Insights