crossplane
Migration should notify of skipped resources
What problem are you facing?
When creating a migration on a local filesystem if a file contains unknown Kinds, the migrator will not process the file without emitting any warnings or errors.
For example, if a manifest contains a Secret which does not have a registered converter, outputs are never generated.
---
apiVersion: v1
kind: Secret
metadata:
name: demo-foo-mq-rabbitmq-creds
namespace: example
annotations:
argocd.argoproj.io/sync-wave: "-2"
type: Opaque
immutable: true
data:
password: "cGFzc3dvcmQK"
---
apiVersion: mq.aws.crossplane.io/v1alpha1
kind: Broker
metadata:
labels:
crossplane.io/provider-version: v0.26.1
name: demo-foo-mq-rabbitmq
namespace: example
annotations:
argocd.argoproj.io/sync-wave: "1"
spec:
forProvider:
region: us-east-1
engineType: RabbitMQ
publiclyAccessible: false
autoMinorVersionUpgrade: false
engineVersion: 3.8.11
hostInstanceType: mq.t3.micro
deploymentMode: SINGLE_INSTANCE
subnetIDs:
- subnet-2cc5ce61
securityGroupIdRefs:
- name: demo-foo-mq-rabbitmq-security-group
users:
- username: rabbitmq-admin
consoleAccess: true
passwordSecretRef:
key: password
name: demo-foo-mq-rabbitmq-creds
namespace: example
tags:
App: crossplane
Environment: qa
Name: crossplane-mq
Product: crossplane
Squad: sandbox
Tier: backend
VS: sandbox
cluster: mq-cluster
writeConnectionSecretToRef:
name: demo-foo-mq-creds
namespace: example
providerConfigRef:
name: awsconfig
The generated migration plan is mostly empty and none of the other directories are crated.
spec: {}
version: 0.1.0
How could Upjet help solve your problem?
The user should be informed of any local filesystem resources that have been skipped during processing.
Another way to create this is have a resource with an error in a file with multiple MRs to convert.
For example, if you have 2 Queues and the 2nd has an error, only the first will be processed. Only when removing the first resource will the error appear:
---
apiVersion: sqs.aws.crossplane.io/v1beta1
kind: Queue
metadata:
labels:
crossplane.io/provider-version: v0.26.1
name: sqs-simple-helm-2-dlq
annotations:
argocd.argoproj.io/sync-wave: "4"
spec:
forProvider:
region: us-east-1
delaySeconds: 4
maximumMessageSize: 262144
messageRetentionPeriod: 345600
receiveMessageWaitTimeSeconds: 0
visibilityTimeout: 30
sseEnabled: true
policy: |
{
"Version": "2012-10-17",
"Id": "sqs-simple-helm-2",
"Statement": [
{
"Sid": "BasicSQS",
"Effect": "Allow",
"Principal": "*",
"Action": [
"sqs:DeleteMessage",
"sqs:SendMessage"
],
"Resource": "arn:aws:sqs:us-east-1::sqs-simple-helm-2",
"Condition": {
"StringLikeIfExists": {
"aws:SourceAccount": ""
}
}
}
]
}
tags:
Name: sqs-simple-helm-2-dlq
VS: sandbox
providerConfigRef:
name: awsconfig
---
apiVersion: sqs.aws.crossplane.io/v1beta1
kind: Queue
metadata:
labels:
crossplane.io/provider-version: v0.26.1
name: sqs-simple-helm-2
namespace:
annotations:
argocd.argoproj.io/sync-wave: "5"
spec:
forProvider:
region: us-east-1
delaySeconds: 4
maximumMessageSize: 262144
messageRetentionPeriod: 345600
receiveMessageWaitTimeSeconds: 0
visibilityTimeout: 30
sseEnabled: true
policy: |
{
"Version": "2012-10-17",
"Id": "sqs-simple-helm-2",
"Statement": [
{
"Sid": "BasicSQS",
"Effect": "Allow",
"Principal": "*",
"Action": [
"sqs:DeleteMessage",
"sqs:SendMessage"
],
"Resource": "arn:aws:sqs:us-east-1::sqs-simple-helm-2",
"Condition": {
"StringLikeIfExists": {
"aws:SourceAccount": ""
}
}
}
]
}
redrivePolicy:
deadLetterTargetArnRef:
name: sqs-simple-helm-2-dlq
maxReceiveCount: 1
tags:
Name: sqs-simple-helm-2
VS: sandbox
providerConfigRef:
name: awsconfig
go run cmd/migrator/main.go -s ./source Using filesystem source main: error: Failed to generate the migration plan: failed to generate the migration plan: failed to migrate resource: failed to migrate resource: failed to copy source into target: failed to convert from unstructured.Unstructured to the managed resource type: unrecognized type: string
I think this issue is also related with https://github.com/upbound/upjet/issues/233
