compose-cli
compose-cli copied to clipboard
docker
Error "InvalidParameterException: Invalid identifier: Identifier is for cluster..." when running docker compose up for AWS ECS
Description
Using AWS ECS, when running "docker compose up", creation of the Service fails with error "InvalidParameterException: Invalid identifier: Identifier is for cluster Test1. Your cluster is default". The cluster defined in the docker compose file is Test1.
Here's the command used to deploy the stack:
> docker -c aws-context compose -f test.yml up
[+] Running 13/13
⠿ jaredhess DeleteComplete 210.0s
⠿ WebTaskRole DeleteComplete 161.0s
⠿ WebTaskExecutionRole DeleteComplete 161.0s
⠿ CloudMap DeleteComplete 205.0s
⠿ WebTCP80TargetGroup DeleteComplete 160.0s
⠿ DefaultNetwork DeleteComplete 159.0s
⠿ LogGroup DeleteComplete 161.0s
⠿ WebTCP80Listener DeleteComplete 157.0s
⠿ Default80Ingress DeleteComplete 53.0s
⠿ DefaultNetworkIngress DeleteComplete 53.0s
⠿ WebTaskDefinition DeleteComplete 144.0s
⠿ WebServiceDiscoveryEntry DeleteComplete 109.0s
⠿ WebService DeleteComplete 104.0s
InvalidParameterException: Invalid identifier: Identifier is for cluster Test1. Your cluster is default
Here's the content of the docker compose file (we have to add the permissions boundary to the roles due to organization policy):
version: "3.8"
x-aws-vpc: "vpc-<redacted>"
x-aws-cluster: "Test1"
x-aws-loadbalancer: "arn:aws:elasticloadbalancing:us-west-2:<redacted>"
services:
web:
image: nginx
ports:
- "80:80"
x-aws-policies:
- "arn:aws:iam::aws:policy/service-role/<redacted>"
x-aws-cloudformation:
Resources:
WebTaskExecutionRole:
Properties:
PermissionsBoundary: "arn:aws:iam::<redacted>"
WebTaskRole:
Properties:
PermissionsBoundary: "arn:aws:iam::<redacted>"
Output of docker-compose --version:
docker-compose version 1.29.2, build 5becea4c
Output of docker version:
Client:
Cloud integration: 1.0.17
Version: 20.10.8
API version: 1.41
Go version: go1.16.6
Git commit: 3967b7d
Built: Fri Jul 30 19:55:20 2021
OS/Arch: darwin/amd64
Context: aws-context
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.8
API version: 1.41 (minimum version 1.12)
Go version: go1.16.6
Git commit: 75249d8
Built: Fri Jul 30 19:52:10 2021
OS/Arch: linux/amd64
Experimental: true
containerd:
Version: 1.4.9
GitCommit: e25210fe30a0a703442421b0f60afac609f950a3
runc:
Version: 1.0.1
GitCommit: v1.0.1-0-g4144b63
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Output of docker context show:
> docker context show
aws-context
> docker context inspect aws-context
[
{
"Name": "aws-context",
"Metadata": {
"Type": "ecs"
},
"Endpoints": {
"docker": {
"SkipTLSVerify": false
},
"ecs": {
"Profile": "default"
}
},
"TLSMaterial": {},
"Storage": {
"MetadataPath": "<redacted>",
"TLSPath": "<redacted>"
}
}
]
Output of docker info (had to change context back to default):
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
compose: Docker Compose (Docker Inc., v2.0.0-rc.3)
scan: Docker Scan (Docker Inc., v0.8.0)
Server:
Containers: 34
Running: 13
Paused: 0
Stopped: 21
Images: 86
Server Version: 20.10.8
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: e25210fe30a0a703442421b0f60afac609f950a3
runc version: v1.0.1-0-g4144b63
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.47-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 1.938GiB
Name: docker-desktop
ID: QW3A:OLNX:63GE:36EZ:V6D5:ME4Q:JVWO:JIH7:I5BK:QRBT:7SFR:RHGP
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional environment details (AWS ECS, Azure ACI, local, etc.): AWS ECS
I'll also note that the service will deploy successfully by using "docker compose convert" to get a cloudformation template, then deploying the template with "aws cloudformation createstack".
Same issue on our side, also.
But, if using docker compose up -d then it works.
Same problem here. Using both x-aws-cluster and x-aws-vpc, or just use either result in the same problem
Same error here, setting x-aws-cluster gives me the InvalidParameterException: Invalid identifier: Identifier is for cluster Demo-Apps. Your cluster is default error
What's weired is that it works when I run the same command with the -d option.
so docker --context "ecs" compose --file ecs.docker-compose.yaml up doesnt work
whereas docker --context "ecs" compose --file ecs.docker-compose.yaml up -d work (daemonized)
I'm also having this problem. Detaching seems to be the solution for now.
Same here:
docker compose up error InvalidParameterException: Invalid identifier: Identifier is for cluster MyECS. Your cluster is default
docker compose up -d works fine.
Looks very buggy and basic alfa version, far away from ready to prod due to lack of customization.
Hello all. For those of you still into using docker-compose to define your services and deploy to ECS, I'd suggest having a look at ECS Compose-X.
Specifically around this issue, the way Compose-X deals with x-cluster to define the ECS Cluster to use if very versatile. You can create a brand new one, or use an existing one.
When using Lookup to use an existing one (which supports cross-accounts lookup for those who want different accounts in CICD), it will also collect settings about the cluster configuration, for example, capacity providers.
If your cluster did not have the FARGATE capacity providers for example, it would autocorrect/override the deployment settings to be compatible with the available runtime. Equally, compose-x allows you to define the compute platform to use for each services individually (see these for reference).
It also supports to set services to run on ECS Anywhere for those of you who would be running ECS instances outside of AWS regions :)
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
I'm also having this issue. Don't really want to add yet another compose plugin into my project (sorry Compose-X). Any idea of if this will get fixed?
I'm also having this issue. Don't really want to add yet another compose plugin into my project (sorry Compose-X). Any idea of if this will get fixed?
No worries :) Good luck with your ECS Journey
This issue has been automatically marked as not stale anymore due to the recent activity.
bumping again. @gtardif @ndeloof @rumpl any chance of this getting looked at?
Same issue here. Cannot run detached either since it is part of a CI/CD pipeline.
FYI I've opened a patch for this via #2269, but given the imminent EOL (#2258, docker/compose-ecs#7), it will (most likely) not be merged in this repository, so I've opened docker/compose-ecs#19 to track this issue, and docker/compose-ecs#20 to patch it in the new repo.
