Unable to access metadata - Cannot connect to Docker daemon

[david-dreyer]

I am unable to successfully get task metadata from the v3 metadata uri due to a failure to connect to the docker.sock file.

In order to avoid any issues stemming from my application itself, I opened a terminal in the ecs-local-endpoints container, and tested some curl commands. The output matches what I see in the logs when I call from my application.

sh-4.2# curl http://169.254.170.2/v3 
Internal Server Error: Failed to list running containers: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

sh-4.2# curl http://169.254.170.2/v3/task
Internal Server Error: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

The creds endpoint works just fine:

sh-4.2# curl http://169.254.170.2/creds
{"AccessKeyId":"XXXXXXXX","Expiration":"2022-12-06T16:58:13Z","RoleArn":"","SecretAccessKey":"XXXXXX","Token":"XXXXX"}

The docker.sock file is properly mapped into the container. And just in case the symlink was an issue, I also tried just mapping the direct path, but it doesn't seem to make a difference.

sh-4.2# ls -l /var/run/docker.sock
lrwxr-xr-x 1 root root 43 Nov 28 20:23 /var/run/docker.sock -> /Users/my.username/.docker/run/docker.sock

## and with direct path mapped ##

sh-4.2# ls -l /var/run/docker.sock
srwxr-xr-x 1 root root 0 Dec  6 16:08 docker.sock

Here is my docker-compose overrides file:

version: "2.4"

networks:
  credentials_network:
    driver: bridge
    ipam:
      config:
        - subnet: "169.254.170.0/24"
          gateway: 169.254.170.1
services:
  ecs-local-endpoints:
    image: amazon/amazon-ecs-local-container-endpoints:latest-arm64
    volumes:
      - /var/run:/var/run
      - $HOME/.aws/:/home/.aws/
    environment:
      HOME: "/home"
      AWS_PROFILE: "default"
    networks:
      credentials_network:
        ipv4_address: "169.254.170.2"

  app:
    depends_on:
      - ecs-local-endpoints
    networks:
      credentials_network:
        ipv4_address: "169.254.170.3"
    environment:
      - AWS_DEFAULT_REGION=us-east-1
      - AWS_REGION=us-east-1
      - AWS_CONTAINER_CREDENTIALS_RELATIVE_URI=/creds
      - ECS_CONTAINER_METADATA_URI_V4=http://169.254.170.2/v3
      - ECS_CONTAINER_METADATA_URI=http://169.254.170.2

I am using the latest version of Docker Desktop -- 4.15.0 (93002) on an M1 Macbook running MacOS 13.0

Is there something obviously wrong here, or is there a known issue either with running on M1 Macs or newer versions of docker?

[ryanjclark]

I also have this issue. Should we be able to curl this?

[ryanjclark]

I changed the volumes to something more explicit and it worked 🤷

volumes:
      - /var/run/docker.sock:/var/run/docker.sock

[dciszewski]

Same here and indeed it worked only when just the file was mounted — you probably want to update blogpost to account for this: https://aws.amazon.com/blogs/compute/a-guide-to-locally-testing-containers-with-amazon-ecs-local-endpoints-and-docker-compose/