Terraform import support

[ShelRoman]

Hello there. Have a question regarding terraform import support, any plans to add this feature? I believe it can be very useful after you folks added terraform support for OSS.

[jace-ys]

I'd like to see this feature as well, otherwise we can't really use Terraform to manage our Airbyte (OSS) instance.

If anyone could point me in the direction, I'm happy to pick this up and help implement it as my first contribution.

[thiagoazcampos]

I was able to import a pre-existing Faker source with import blocks, it worked just fine.

I had some problems while importing more complex resources, but I think it's more related to other issues than importing itself.

[jace-ys]

Interesting. @thiagoazcampos If you don't mind could you share which version of the provider you're using and what the imported state looks like in your tfstate file?

FWIW, my import of a Faker source did succeed but the configuration block of the source had all its fields as null.

[thiagoazcampos]

I'm using the provider on v0.3.4, Terraform on v1.5.6, and Airbyte OSS on version 0.50.29.

Here's the resource and import definitions:

resource "airbyte_source_faker" "my_source_faker" {
  configuration = {
    always_updated    = false
    count             = 3
    parallelism       = 9
    records_per_slice = 5
    seed              = 6
    source_type       = "faker"
  }
  name         = "Faker Source"
  workspace_id = var.workspace_id
}
import {
  to = airbyte_source_faker.my_source_faker
  id = "34a1446e-8cce-4149-9659-45ee9b5b7fd2" # id of the existing source
}

And the resulting state on the tfstate file:

{
      "mode": "managed",
      "type": "airbyte_source_faker",
      "name": "my_source_faker",
      "provider": "provider[\"registry.terraform.io/airbytehq/airbyte\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "configuration": {
              "always_updated": false,
              "count": 3,
              "parallelism": 9,
              "records_per_slice": 5,
              "seed": 6,
              "source_type": "faker"
            },
            "name": "Faker Source",
            "secret_id": null,
            "source_id": "34a1446e-8cce-4149-9659-45ee9b5b7fd2",
            "source_type": "faker",
            "workspace_id": "1bfa108d-486e-4f02-8d7a-6b916224e93b"
          },
          "sensitive_attributes": []
        }
      ]
    }

Although, while trying the import with Terraform generating the resource code, the configuration fields indeed end up as null. Please let me know if you guys were talking about this specific use-case and I got it wrong. 😅

[jace-ys]

Thanks @thiagoazcampos, very helpful. I can see why your example works, as you have the configuration fields specified in your resource block, so the state will reflect that.

However, I believe the provider doesn't actually detect state in the upstream. You can test this by manually making changes in the Airbyte UI (eg. change count). When you run another plan/apply, you will see that the provider doesn't actually read the state from the upstream, so to Terraform it will look like no diff has occurred.

This is not problematic if you're certain changes won't be made manually upstream, but the provider not being able to read the upstream state and detect drift is unideal IMO. This is also the reason why generating the resource code or terraform import doesn't work properly - the actual state of configuration is not being read.

[jace-ys]

Looking at the provider code, I believe the missing piece is that the RefreshFromGetResponse function does not actually populate the configuration field. Potentially something missing/misconfigured in the OpenAPI spec?