awscc_dms_data_provider does not create

[kstephensontm]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment
• The resources and data sources in this provider are generated from the CloudFormation schema, so they can only support the actions that the underlying schema supports. For this reason submitted bugs should be limited to defects in the generation and runtime code of the provider. Customizing behavior of the resource, or noting a gap in behavior are not valid bugs and should be submitted as enhancements to AWS via the CloudFormation Open Coverage Roadmap.

Terraform CLI and Terraform AWS Cloud Control Provider Version

Affected Resource(s)

• awscc_dms_data_provider

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "awscc_dms_data_provider" "dms_data_provider_source" {
  data_provider_name = "sc-sql-server"
  engine = "sqlserver"
  settings = jsonencode({
    MicrosoftSqlServerSettings = {
      DatabaseName = "mssql"
      Port         = 1433
      ServerName   = "mssql-default.blah.us-west-2.rds.amazonaws.com"
      SslMode      = "none"
    }
  })
}

Debug Output

Panic Output

Expected Behavior

The resource awscc_dms_data_provider should have created the data provider. I was able to create the data provider manually and import without any issues which tells me this is a bug. It will not let me create a new data provider.

Actual Behavior

Just errors out with a GeneralServiceException.

│ Error: AWS SDK Go Service Operation Incomplete │ │ with awscc_dms_data_provider.dms_data_provider_source, │ on dms-schema-conversion.tf line 55, in resource "awscc_dms_data_provider" "dms_data_provider_source": │ 55: resource "awscc_dms_data_provider" "dms_data_provider_source" { │ │ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state │ transitioned to FAILED. StatusMessage: Error occurred during operation 'AWS::DMS::CreateDataProvider'.. │ ErrorCode: GeneralServiceException

Steps to Reproduce

1. terraform apply

Important Factoids

None

References

https://registry.terraform.io/providers/hashicorp/awscc/latest/docs/resources/dms_data_provider

[quixoticmonk]

Thank you for opening the issue, @kstephensontm . Can you check with the settings based on the schema below ?

resource "awscc_dms_data_provider" "dms_data_provider_source" {
  data_provider_name = "sc-sql-server"
  engine = "sqlserver"
  settings = {
    microsoft_sql_server_settings = {
      database_name = "mssql"
      port         = 1433
      server_name   = "mssql-default.blah.us-west-2.rds.amazonaws.com"
      ssl_mode      = "none"
    }
  }
}

[kstephensontm]

It gives me this error....

│ Error: Incorrect attribute value type │ │ on dms-schema-conversion.tf line 29, in resource "awscc_dms_data_provider" "dms_data_provider_source": │ 29: settings = { │ 30: microsoft_sql_server_settings = { │ 31: database_name = "mssql" │ 32: port = 1433 │ 33: server_name = "mssql-default.blah.us-west-2.rds.amazonaws.com" │ 34: ssl_mode = "none" │ 35: } │ 36: } │ │ Inappropriate value for attribute "settings": string required.

[quixoticmonk]

What version of the awscc provider are you using ? I am not able to reproduce your scenario.

Plan: 1 to add, 0 to change, 0 to destroy.
awscc_dms_data_provider.dms_data_provider_source: Creating...
awscc_dms_data_provider.dms_data_provider_source: Still creating... [10s elapsed]
awscc_dms_data_provider.dms_data_provider_source: Creation complete after 11s [id=arn:aws:dms:us-east-1:XXXX:data-provider:QB3OSOECCBDZPFZP2VCKWX2MVU]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

[kstephensontm]

the latest version ...

awscc = {
  source  = "hashicorp/awscc"
  version = "0.74.0"
}

[quixoticmonk]

That must be it. The version has been bumped and from the attribute reference I see the difference in schema. 0.75

[settings](https://registry.terraform.io/providers/hashicorp/awscc/0.75.0/docs/resources/dms_data_provider#settings) 

(Attributes) The property identifies the exact type of settings for the data provider. (see [below for nested schema](https://registry.terraform.io/providers/hashicorp/awscc/0.75.0/docs/resources/dms_data_provider#nestedatt--settings))

0.74

[settings](https://registry.terraform.io/providers/hashicorp/awscc/0.74.0/docs/resources/dms_data_provider#settings) 

(String) The property identifies the exact type of settings for the data provider.

[kstephensontm]

Yes, that worked. Wow, they just published it yesterday after I tried all of this. Thanks for your help.

[kstephensontm]

hi @quixoticmonk ... i'm creating a provider for postgresql aurora and it's saying it's not supported but i'm able to create it manually with no issues. any ideas on why it's throwing this error?

│ Error: AWS SDK Go Service Operation Incomplete │ │ with awscc_dms_data_provider.dms_data_provider_target, │ on dms-schema-conversion.tf line 39, in resource "awscc_dms_data_provider" "dms_data_provider_target": │ 39: resource "awscc_dms_data_provider" "dms_data_provider_target" { │ │ Waiting for Cloud Control API service CreateResource operation completion returned: waiter │ state transitioned to FAILED. StatusMessage: DMS doesn't support aurora_postgresql database │ engine. Choose another database engine and try again. (Service: DatabaseMigration, Status │ Code: 400, Request ID: a5033be9-0e1c-457a-bd9a-e751af7e224f). ErrorCode: InvalidRequest

resource "awscc_dms_data_provider" "dms_data_provider_target" { data_provider_name = "sc-postgresql2" engine = "aurora_postgresql" settings = { postgre_sql_settings = { database_name = "pssql" port = 5432 server_name = "postgres-default-1.blah.us-west-2.rds.amazonaws.com" ssl_mode = "none" } } }

[kstephensontm]

also ... i have my sqlserver server name set with blah and it works. as soon as i change it to the actual endpoint i get the following error.

server_name = "mssql-default.blah.us-west-2.rds.amazonaws.com"

│ Error: AWS SDK Go Service Operation Incomplete │ │ with awscc_dms_data_provider.dms_data_provider_source, │ on dms-schema-conversion.tf line 26, in resource "awscc_dms_data_provider" "dms_data_provider_source": │ 26: resource "awscc_dms_data_provider" "dms_data_provider_source" { │ │ Waiting for Cloud Control API service UpdateResource operation completion returned: waiter │ state transitioned to FAILED. StatusMessage: Error occurred during operation │ 'AWS::DMS::UpdateDataProvider'.. ErrorCode: GeneralServiceException

[kstephensontm]

just realized that the error of changing the sqlserver to the actual endpoint errored because that endpoint is already in use. Would be nice to have the error state that though. I'm still having issues with the engine being set to aurora_postgresql. this says it's not supported.

[quixoticmonk]

I can reproduce the engine reference for aurora_postgreqsl.
@kstephensontm Do you mind opening another issue for the GeneralServiceException on the endpoint change?

│ Error: AWS SDK Go Service Operation Incomplete
│
│   with awscc_dms_data_provider.dms_data_provider_source,
│   on main.tf line 1, in resource "awscc_dms_data_provider" "dms_data_provider_source":
│    1: resource "awscc_dms_data_provider" "dms_data_provider_source" {
│
│ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state
│ transitioned to FAILED. StatusMessage: DMS doesn't support aurora_postgresql database engine.
│ Choose another database engine and try again. (Service: DatabaseMigration, Status Code: 400,

@wellsiau-aws This might be another upstream one.

I do get the valid list from the api when I try with an incorrect engine type.

 Error: Invalid Attribute Value Match
│
│   with awscc_dms_data_provider.dms_data_provider_source,
│   on main.tf line 3, in resource "awscc_dms_data_provider" "dms_data_provider_source":
│    3:   engine = "aurora-postgresql"
│
│ Attribute engine value must be one of: ["postgres" "mysql" "oracle" "sqlserver" "aurora"
│ "aurora_postgresql"], got: "aurora-postgresql"

[kstephensontm]

created issue https://github.com/hashicorp/terraform-provider-awscc/issues/1636

[wellsiau-aws]

@kstephensontm @quixoticmonk , thanks for reporting this issue. I will respond about the engine selection problem on #1636

with that said, are there still outstanding problem that we need to cover on this particular issue?

[gzurowski]

@wellsiau-aws I am getting similar issues when trying to create a data provider for a Postgres database using the following code:

resource "awscc_dms_data_provider" "source" {
  data_provider_name = "my-data-provider-source"
  engine             = "postgres"
  settings = {
    postgre_sql_settings = {
      server_name   = var.source_database_hostname
      port          = 5432
      database_name = "mydatabase1"
      ssl_mode      = "require"
    }
  }
}

The error looks as follows:

module.database-migraton.awscc_dms_data_provider.source: Creating...
╷
│ Error: AWS SDK Go Service Operation Incomplete
│
│   with module.database-migraton.awscc_dms_data_provider.source,
│   on ../../../modules/database-migration/main-homogeneous.tf line 1, in resource "awscc_dms_data_provider" "source":
│    1: resource "awscc_dms_data_provider" "source" {
│
│ Waiting for Cloud Control API service CreateResource operation completion returned: waiter state transitioned to FAILED. StatusMessage: Error occurred during operation
│ 'AWS::DMS::CreateDataProvider'.. ErrorCode: GeneralServiceException

[quixoticmonk]

@gzurowski I was able to create the resource without any issues on my end for postgres. What version of tf cli, provider are you on ?

resource "awscc_dms_data_provider" "dms_data_provider_target" {
  data_provider_name = "sc-postgresql2"
  engine             = "postgres"
  settings = {
    postgre_sql_settings = {
      database_name = "pssql"
      port          = 5432
      server_name   = "postgres-default-1.blah.us-west-2.rds.amazonaws.com"
      ssl_mode      = "require" # Tried both none and require and resource was created without any issues
    }
  }
}

[kstephensontm]

postgres works for me, it's aurora_postgresql that does not work.

[kstephensontm]

i have this issue open for it. https://github.com/hashicorp/terraform-provider-awscc/issues/1636

[gzurowski]

@gzurowski I was able to create the resource without any issues on my end for postgres. What version of tf cli, provider are you on ?

I am using the following version (and the creation of the data provider fails consistently):

Terraform v1.7.5
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.47.0
+ provider registry.terraform.io/hashicorp/awscc v0.75.0