[Bug]: hcloud_volume_attachment replaced on subsequent runs

[symptog]

What happened?

Terraform replaces the hcloud_volume_attachment on every subsequent Terraform apply (but not on the very first) even if nothing changed within the Terraform script. As a result the disk/volume is removed from the server and added again.

Terraform only creates the server and the attachment. The volume id is received via a data resource.

Terraform plan:

 # hcloud_volume_attachment.testvolume_attachment must be replaced
-/+ resource "hcloud_volume_attachment" "testvolume_attachment" {
      ~ id        = "1598XXXX" -> (known after apply)
      ~ volume_id = 1598XXXX -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

dmesg after terraform apply

[Fri Jan  7 19:01:10 2022] blk_update_request: I/O error, dev sdb, sector 52481744 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 0
[Fri Jan  7 19:01:10 2022] blk_update_request: I/O error, dev sdb, sector 52481800 op 0x1:(WRITE) flags 0x0 phys_seg 2 prio class 0
[Fri Jan  7 19:01:10 2022] sdb1: writeback error on inode 67160493, offset 0, sector 52481760
[Fri Jan  7 19:01:10 2022] sdb1: writeback error on inode 67160488, offset 0, sector 52481808
[Fri Jan  7 19:01:10 2022] sdb1: writeback error on inode 67160489, offset 0, sector 52481816
[Fri Jan  7 19:01:10 2022] blk_update_request: I/O error, dev sdb, sector 52455688 op 0x1:(WRITE) flags 0x9800 phys_seg 1 prio class 0
[Fri Jan  7 19:01:10 2022] XFS (sdb1): log I/O error -5
[Fri Jan  7 19:01:10 2022] XFS (sdb1): xfs_do_force_shutdown(0x2) called from line 1196 of file fs/xfs/xfs_log.c. Return address = 0000000048b36308
[Fri Jan  7 19:01:10 2022] XFS (sdb1): Log I/O Error Detected. Shutting down filesystem
[Fri Jan  7 19:01:10 2022] XFS (sdb1): Please unmount the filesystem and rectify the problem(s)
[Fri Jan  7 19:01:10 2022] sd 0:0:0:1: [sdb] Synchronizing SCSI cache
[Fri Jan  7 19:01:10 2022] sd 0:0:0:1: [sdb] Synchronize Cache(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[Fri Jan  7 19:01:10 2022] sd 0:0:0:1: [sdb] Sense Key : Illegal Request [current] 
[Fri Jan  7 19:01:10 2022] sd 0:0:0:1: [sdb] Add. Sense: Logical unit not supported
[Fri Jan  7 19:01:12 2022] scsi 0:0:0:1: Direct-Access     HC       Volume           2.5+ PQ: 0 ANSI: 5
[Fri Jan  7 19:01:12 2022] sd 0:0:0:1: Attached scsi generic sg1 type 0
[Fri Jan  7 19:01:12 2022] sd 0:0:0:1: Power-on or device reset occurred
[Fri Jan  7 19:01:12 2022] sd 0:0:0:1: [sdc] 104857600 512-byte logical blocks: (53.7 GB/50.0 GiB)
[Fri Jan  7 19:01:12 2022] sd 0:0:0:1: [sdc] Write Protect is off
[Fri Jan  7 19:01:12 2022] sd 0:0:0:1: [sdc] Mode Sense: 63 00 00 08
[Fri Jan  7 19:01:12 2022] sd 0:0:0:1: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[Fri Jan  7 19:01:12 2022]  sdc: sdc1
[Fri Jan  7 19:01:12 2022] sd 0:0:0:1: [sdc] Attached SCSI disk

What did you expect to happen?

I expect the hcloud_volume_attachment not to change.

Please provide a minimal working example

Create disk per API (httpie)

cat <<EOF > volume.json
{
  "automount": false,
  "format": "xfs",
  "location": "fsn1",
  "name": "testvolume",
  "size": 50
}
EOF

http POST https://api.hetzner.cloud/v1/volumes "Authorization: Bearer $HCLOUD_TOKEN" < volume.json

resource "hcloud_server" "testserver" {
  name        = "test-server"
  image       = "debian-11"
  location    = "fsn1"
  server_type = "cpx11"
  ssh_keys    = [ "mysshkey" ]
}

# Need to be created upfront
data "hcloud_volume" "testvolume" {
  name      = "testvolume"
}

resource "hcloud_volume_attachment" "testvolume_attachment" {
  volume_id = data.hcloud_volume.testvolume.id
  server_id = hcloud_server.testserver.id
  automount = false
}

[fhofherr]

Hi and thanks for your issue report including the detailed example. Unfortunately it seems I cannot reproduce the issue. No matter how often I execute terraform plan I always get:

hcloud_server.testserver: Refreshing state... [id=17174919]
hcloud_volume_attachment.testvolume_attachment: Refreshing state... [id=16052438]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

terraform apply works as expected as well:

hcloud_server.testserver: Refreshing state... [id=17174919]
hcloud_volume_attachment.testvolume_attachment: Refreshing state... [id=16052438]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Just to make sure I didn't mix anything up, this is the terraform configuration I used:

terraform {
  required_providers {
    hcloud = {
      source     = "hetznercloud/hcloud"
      version    = "1.32.2"
    }
  }
}

resource "hcloud_server" "testserver" {
  name        = "test-server"
  image       = "debian-11"
  location    = "fsn1"
  server_type = "cpx11"
  ssh_keys    = [ "<ssh-key>" ]
}

# Need to be created upfront
data "hcloud_volume" "testvolume" {
  name      = "testvolume"
}

resource "hcloud_volume_attachment" "testvolume_attachment" {
  volume_id = data.hcloud_volume.testvolume.id
  server_id = hcloud_server.testserver.id
  automount = false
}

The only differences to your example are the name of the SSH key and the provider selection. Could you please check which version of the Terraform provider you are using?

[symptog]

I've seen that our volume is protected. maybe this is the issue. I'll retry again.

[symptog]

ok. The problem seems to be our GitLab Backend for Terraform state. The server_id in the volume doesn't reflect the correct state.

    {
      "module": "module.hedgedoc",
      "mode": "data",
      "type": "hcloud_volume",
      "name": "hedgedoc_data",
      "provider": "provider[\"registry.terraform.io/hetznercloud/hcloud\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "delete_protection": true,
            "id": 1598xxxx,
            "labels": {},
            "linux_device": "/dev/disk/by-id/scsi-0HC_Volume_1598xxxx",
            "location": "fsn1",
            "name": "hedgedoc-data",
            "selector": null,
            "server_id": null,
            "size": 50,
            "with_selector": null,
            "with_status": null
          },
          "sensitive_attributes": []
        }
      ]
    },

I'm currently tying to figure out if this is a problem with your provider or Terraform itself.

[symptog]

It works if I use the data resource with the id attribute

# Need to be created upfront
data "hcloud_volume" "testvolume" {
  id = "123456"
}

Don't ask me why ^^

[apricote]

I too am unable to reproduce this issue. I am going to close the issue for now, if you still have this problem, feel free to reopen or create a new issue.

If you still encounter this issue, it would be great if you can send us a debug log for the run: TF_LOG=DEBUG TF_LOG_PATH=volume-attachment-replace.log terraform apply ....