copilot-cli icon indicating copy to clipboard operation
copilot-cli copied to clipboard

[Bug]: Send sidecar logs to aws-fluent-bit also

Open proof-nicholas opened this issue 11 months ago • 0 comments

Description:

I have defined the aws-fluent-bit logging for my service and the logs for the main container are published to my configured log destination as expected but it appears that any sidecars defined in the service are not sent to aws-fluent-bit but instead are sent to the default awslogs driver. How can I ensure all my containers or at least configure a subset of my sidecards to use the aws-fluent-bit logging?

Details:

Example:

name: api
type: "Load Balanced Web Service"

logging:
  image: aws-fluent-bit
  destination:
    Name: "datadog"
    Host: "http-intake.logs.datadoghq.com"
    compress: "gzip"
    dd_service: "api"
    dd_source: "api"
    dd_tags: "env:${COPILOT_ENVIRONMENT_NAME}"
    TLS: "on"
    provider: "ecs"
  secretOptions:
    apiKey: XXXXXX
  configFilePath: "/fluent-bit/configs/parse-json.conf"

sidecars:
  datadog-agent:
    image:
      location: public.ecr.aws/datadog/agent:latest
    secrets:
      DD_API_KEY: XXXXXX
    variables:
      ECS_FARGATE: true
      DD_PROCESS_AGENT_PROCESS_COLLECTION_ENABLED: true
      DD_DOGSTATSD_NON_LOCAL_TRAFFIC: true
  mysidecar:
    port: 23853
    image:
      location: XXXXXXX

image:
  location: XXXXX/api
  port: 8080
  depends_on:
    mysidecar: healthy

Observed result:

Logs for only the main api container are available in Datadog. All other logs are sent to CloudWatch.

Expected result:

Logs for mysidecar container are also available in Datadog.

Debugging:

proof-nicholas avatar Jan 09 '25 13:01 proof-nicholas