aws_iot_topic_rule can't update kafka action

[cagriekin]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.1.9
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.13.0
+ provider registry.terraform.io/pagerduty/pagerduty v2.2.1

Affected Resource(s)

• aws_iot_topic_rule

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_iot_topic_rule" "kafka-rule" {
  name        = "iot_machine_event"
  enabled     = true
  sql         = "SELECT * FROM 'any_topic'"
  sql_version = "2016-03-23"

  kafka {
    client_properties = {
      "acks"                = "1"
      "bootstrap.servers"   = "bootstrap_servers"
      "compression.type"    = "lz4"
      "security.protocol"   = "SASL_SSL"
      "sasl.mechanism"      = "SCRAM-SHA-512"
      "sasl.scram.username" = "$${get_secret('secret_name', 'SecretString', 'username', 'role_arn')}"
      "sasl.scram.password" = "$${get_secret('secret_name', 'SecretString', 'password', 'role_arn')}"
    }
    topic           = "destination_topic"
    destination_arn = aws_iot_topic_rule_destination.vpc-destination.arn
  }
}

Debug Output

https://gist.github.com/cagriekin/72f55f1b3c6392407be70487eaa83d2a

Panic Output

╷
│ Error: replacing IoT Topic Rule (iot_machine_event): InvalidParameter: 3 validation error(s) found.
│ - missing required field, ReplaceTopicRuleInput.TopicRulePayload.Actions[1].Kafka.ClientProperties.
│ - missing required field, ReplaceTopicRuleInput.TopicRulePayload.Actions[1].Kafka.DestinationArn.
│ - missing required field, ReplaceTopicRuleInput.TopicRulePayload.Actions[1].Kafka.Topic.
│
│
│   with aws_iot_topic_rule.machine-events,
│   on iot-event-rule.tf line 72, in resource "aws_iot_topic_rule" "machine-events":
│   72: resource "aws_iot_topic_rule" "machine-events" {
│
╵
Releasing state lock. This may take a few moments...

Expected Behavior

Should update the iot topic rule with new kafka action changes.

Actual Behavior

Outputs an error

Steps to Reproduce

1. Create a rule containing a kafka action
2. Plan & Apply changes
3. Change anything in client_properties, for example compression type
4. Plan & Apply Changes

Important Factoids

References

[sdhuang32]

Hey Terraform team,

I just encountered the exact same issue (using AWS provider v4.33.0). Wondering if there's any update or schedule to solve this?

Hey @cagriekin , did you find any magic to workaround this problem?

[cdsre]

Can confirm this is still an issue in hashicorp/aws v4.55.0. The issue is not triggered by only changes in client properties. I have encountered this issue when changing for example the key for the kafka topic which sits in the kafka block but outside the client_properties

[DevOpsDerek]

We are seeing this issue as well. Is this still a problem?

[camm73]

We are also still seeing this issue. Is there a timeline for this fix?

[Jyoti492]

we also see the same. is there any solution for the same ?

[mbbush]

Has anyone found any workarounds? Does CloudFormation manage iot topic rules properly? Maybe I could manage a CloudFormation stack with terraform?

[RockyMM]

As mentioned in https://github.com/upbound/provider-aws/issues/748, the problem is that clientProperties has no schema and no changes are noticed if there are changes only on client_properties. The workaround is to introduce trivial change on any other property, e.g., name.

[mbbush]

@RockyMM I added a test for only updating part of client_properties to my PR that fixes this issue, and I don't see any different behavior between changes inside or outside of client_properties.

[github-actions[bot]]

This functionality has been released in v5.16.2 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.