[Enhancement]: Allow MSK to upgrade to tiered storage from local storage

[corserIBM]

Description

Not sure if this should be an enhancement or defect but would like to see the provider be able to upgrade an aws_msk_cluster set up with LOCAL storage mode to TIERED storage.

Affected Resource(s) and/or Data Source(s)

aws_msk_cluster

Potential Terraform Configuration

No change to existing configuration

References

No response

Would you like to implement a fix?

No

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[justinretzolk]

Hey @corserIBM 👋 Thank you for taking the time to raise this! It looks like the argument is already in place to do this -- storage_mode -- so I would expect that this functionality is already possible, assuming that you meet the requirements laid out in the AWS documentation on transitioning to tiered storage. Are you seeing behavior outside of this? If so, can you supply a sample Terraform configuration that can reproduce the issue, as well as debug logs (redacted as needed)?

[corserIBM]

Attached terraform config and log. The config was initially applied with storage_mode = "LOCAL" and when updated to TIERED it looks like the apply is going to update the cluster but it only takes a second to apply the change and then in the UI it's still only EBS and no cluster operations.

Through the CLI the operation appears immediately

aws kafka update-storage --cluster-arn $CLUSTER_ARN --storage-mode TIERED --current-version $CURRENT_VERSION

main.tf.txt cluster.log

[vrecan]

I'm seeing the same issue when updating storage_mode from "LOCAL" -> "TIERED".

[py-zoid]

Same issue, terraform run shows the change has been applied but the AWS UI doesn't reflect the same and still shows, "EBS storage only".

[mgusiew-guide]

I am observing same issue on AWS provider 5.7.0. When updating "LOCAL" -> "TIERED", the Terraform plan shows update on storage mode from "LOCAL" -> "TIERED", apply operation is successful but the cluster storage mode is still set to LOCAL.

[kia-martinez]

Same issue here. On AWS Provider 4.4, but when when i temporarily upgraded to the most recent the issue persists. I had to change in the console and only then does terraform not show a diff.

[thestuckster]

Is there any update to this issue? We are seeing the same issue when updating from LOCAL to TIERED using MSK Version 2.8.2.tiered

[AlexeyRaga]

Just for the record: same issue here

[bryanwieger]

I am having the same issue as well.

Looks like the change to code should be around here https://github.com/hashicorp/terraform-provider-aws/blob/5af43cb71d752074dc6bbdf965ce12f603eb6f42/internal/service/kafka/cluster.go#L721

[zandieh]

I'm also facing the same issue!

[sehorne]

The resourceClusterUpdate() seems to be missing a if d.HasChange("storage_mode") {} at the cluster level for the storage mode. See existing reference in the create call: https://github.com/hashicorp/terraform-provider-aws/blob/9cc91b0de9ebc5aa8ed1843e36f29ec54e97e34b/internal/service/kafka/cluster.go#L713

See api call here: https://docs.aws.amazon.com/msk/1.0/apireference/clusters-clusterarn-storage.html