community
community copied to clipboard
aws-controllers-k8s
Create a MQ broker in non-default vpc
Describe the bug I'm not able to create a broker in a different VPC than the default one
Steps to reproduce Create an MQ broker using the following manifest: apiVersion: mq.services.k8s.aws/v1alpha1 kind: Broker metadata: name: my-broker spec: name: my-broker autoMinorVersionUpgrade: false deploymentMode: SINGLE_INSTANCE engineType: RabbitMQ engineVersion: "3.9.16" hostInstanceType: "mq.t3.micro" publiclyAccessible: false
storageType: string
securityGroups:
- SECURITY_GROUP_ID subnetIDs:
- SUBNET_ID_GROUP_ID
users:
- password: namespace: ack-system name: mq-password key: password groups: [] consoleAccess: true username: admin => 2022-10-21T22:50:06.756Z ERROR controller.broker Reconciler error {"reconciler group": "mq.services.k8s.aws", "reconciler kind": "Broker", "name": "my-broker", "namespace": "ack-system", "error": "BadRequestException: Subnets must belong to the same VPC.\n{\n RespMetadata: {\n StatusCode: 400,\n RequestID: "e32fab71-2318-4a95-bcac-b1fcc27c86df"\n },\n ErrorAttribute: "subnetIds",\n Message_: "Subnets must belong to the same VPC."\n}"}
I think there is need to add VPCId ine the mq broker spec
Expected outcome create an Broker in VPC different from the default one
Environment
- Kubernetes version:1.23
- Using EKS (yes/no), if so version? eks.2
- AWS service targeted (S3, RDS, etc.) MQ
Hi @chakch are you able reproduce this issue using CDK or aws cli? just trying to understand if this ACK related or an API behaviour
I'm able to create a broker in a specific vpc other than the default one. the requirement is that if you specify a security group from a vpc you also need to specify the subnet ids on that vpc.
Here is a yaml that I used to create a broker
apiVersion: mq.services.k8s.aws/v1alpha1
kind: Broker
metadata:
name: mq-eks-workshop
spec:
name: mq-eks-workshop
deploymentMode: SINGLE_INSTANCE
engineType: ActiveMQ
engineVersion: "5.15.8"
hostInstanceType: "mq.t3.micro"
publiclyAccessible: false
autoMinorVersionUpgrade: false
users:
- password:
namespace: default
name: mq-eks-workshop
key: password
groups: []
consoleAccess: true
username: admin
subnetIDs:
- $(VPC_PRIVATE_SUBNET_ID_0)
securityGroupRefs:
- from:
name: mq-eks-workshop
/close
@csantanapr: You can't close an active issue/PR unless you authored it or you are a collaborator.
In response to this:
I'm able to create a broker in a specific vpc other than the default one. the requirement is that if you specify a security group from a vpc you also need to specify the subnet ids on that vpc.
Here is a yaml that I used to create a broker
apiVersion: mq.services.k8s.aws/v1alpha1 kind: Broker metadata: name: mq-eks-workshop spec: name: mq-eks-workshop deploymentMode: SINGLE_INSTANCE engineType: ActiveMQ engineVersion: "5.15.8" hostInstanceType: "mq.t3.micro" publiclyAccessible: false autoMinorVersionUpgrade: false users: - password: namespace: default name: mq-eks-workshop key: password groups: [] consoleAccess: true username: admin subnetIDs: - $(VPC_PRIVATE_SUBNET_ID_0) securityGroupRefs: - from: name: mq-eks-workshop/close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@chakch I didn't read closely the first time. I was able to create a ActiveMQ specifying subnet and security-group from a non default vpc and worked, but now I tried to create a RabbitMQ and I can't create it on a none-default vpc
@A-Hilaly here is a yaml with specifying subnet and security-group for rabbitmq
apiVersion: mq.services.k8s.aws/v1alpha1
kind: Broker
metadata:
name: mq-eks-workshop
spec:
name: mq-eks-workshop
deploymentMode: SINGLE_INSTANCE
engineType: RabbitMQ
engineVersion: "3.10.10"
hostInstanceType: "mq.t3.micro"
publiclyAccessible: false
autoMinorVersionUpgrade: false
users:
- password:
namespace: default
name: mq-eks-workshop
key: password
groups: []
consoleAccess: true
username: admin
subnetIDs:
- subnet-0e95abf6cd1647262
securityGroups:
- sg-0a97c3ca774be1446
The broker will be created fine, you can see it in the console with the in Running state with correct subnet and security group.
In the status of the broker you will see the error
status:
ackResourceMetadata:
ownerAccountID: "123456789"
region: us-east-1
conditions:
- message: |-
BadRequestException: Subnets and security groups must belong to the default VPC when only one is provided.
{
RespMetadata: {
StatusCode: 400,
RequestID: "5ffd35ab-3d65-4776-ade4-cf126af0c997"
},
ErrorAttribute: "subnetIds",
Message_: "Subnets and security groups must belong to the default VPC when only one is provided."
}
status: "True"
type: ACK.Recoverable
- lastTransitionTime: "2022-11-22T01:12:40Z"
message: Unable to determine if desired resource state matches latest observed
state
reason: |-
BadRequestException: Subnets and security groups must belong to the default VPC when only one is provided.
{
RespMetadata: {
StatusCode: 400,
RequestID: "5ffd35ab-3d65-4776-ade4-cf126af0c997"
},
ErrorAttribute: "subnetIds",
Message_: "Subnets and security groups must belong to the default VPC when only one is provided."
}
status: Unknown
type: ACK.ResourceSynced
In Cloud Trail you see the go sdk sending an UpdateBroker call after the Broker is Running and includes in the parameters the security group, which is already set to that value.
"eventTime": "2022-11-22T00:37:42Z",
"eventSource": "amazonmq.amazonaws.com",
"eventName": "UpdateBroker",
"awsRegion": "us-east-1",
"sourceIPAddress": "18.235.50.27",
"userAgent": "aws-controllers-k8s/-v0.0.23 (GitCommit/740eedcdf35da5376ee0df354447476ca02d90f1; BuildDate/2022-11-15T17:22; CRDKind/; CRDVersion/) aws-sdk-go/1.44.93 (go1.17.13; linux; amd64)",
"errorCode": "BadRequestException",
"requestParameters": {
"engineVersion": "3.10.10",
"broker-id": "b-519d2e3f-c154-41c1-8a61-ffcb94c6b9f9",
"securityGroups": [
"sg-0db2eb2c65e795d4b"
],
"autoMinorVersionUpgrade": false,
"hostInstanceType": "mq.t3.micro"
},
"responseElements": {
"message": "Changing security groups is not supported for RabbitMQ brokers.",
"errorAttribute": "securityGroups"
},
"requestID": "2eabed1e-a66d-4613-8192-fedca30002d4",
"eventID": "1f396ee4-df9c-45ce-9079-e22878793a01",
"readOnly": false,
"eventType": "AwsApiCall",
"managementEvent": true,
In the logs
2022-11-22T01:20:45.606Z INFO ackrt desired resource state has changed {"account": "123456789", "role": "", "region": "us-east-1", "kind": "Broker", "namespace": "default", "name": "mq-eks-workshop", "is_adopted": false, "generation": 1, "diff": [{"Path":{"Parts":["Spec","AuthenticationStrategy"]},"A":null,"B":"simple"},{"Path":{"Parts":["Spec","EncryptionOptions"]},"A":null,"B":{"useAWSOwnedKey":true}},{"Path":{"Parts":["Spec","Logs"]},"A":null,"B":{"general":false}},{"Path":{"Parts":["Spec","MaintenanceWindowStartTime"]},"A":null,"B":{"dayOfWeek":"FRIDAY","timeOfDay":"22:00","timeZone":"UTC"}},{"Path":{"Parts":["Spec","StorageType"]},"A":null,"B":"ebs"}]}
2022-11-22T01:20:45.838Z ERROR controller.broker Reconciler error {"reconciler group": "mq.services.k8s.aws", "reconciler kind": "Broker", "name": "mq-eks-workshop", "namespace": "default", "error": "BadRequestException: Changing security groups is not supported for RabbitMQ brokers.\n{\n RespMetadata: {\n StatusCode: 400,\n RequestID: \"a1e5ae66-86be-4624-8dc9-8dc631e6037f\"\n },\n ErrorAttribute: \"securityGroups\",\n Message_: \"Changing security groups is not supported for RabbitMQ brokers.\"\n}"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227
I think having the Broker condition ACK.ResourceSynced not True affects the fieldexporters .
status: Unknown
type: ACK.ResourceSynced
It looks to fix this we need to find a way to avoid to include security group on this API call, or avoid the API call since the values are the same that already set on the Broker.
Looks like we have multiple issues in here:
- Missing late initialization.
- Delta function could be improved.
- We should not try to update field that did not change.
@A-Hilaly any updates on this issue, I want to be able to show users how to use ACK with rabbitmq and currently is not working
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale
Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle rotten
Issues go stale after 180d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 60d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale
