Importing a ServiceAccount fails because Secret creation timestamp differs "too much"

[erSitzt]

Importing an existing ServiceAccount fails because the Secret creationDate differs too much.

kubernetes_service_account.projekt_serviceaccounts["eec"]: Importing from ID "eec/eec"...

Error: Failed to discover the default service account token: Unable to find any service accounts tokens which could have been the default one

Terraform Version and Provider Version

Terraform v0.12.25

• provider.kubernetes v1.11.2

Affected Resource(s)

• kubernetes_service_account

Debug Output

I cut some unnecessary parts out...

kubernetes_service_account.projekt_serviceaccounts["service"]: Importing from ID "service/service"...
2020-05-19T13:08:22.816+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: 2020/05/19 13:08:22 [DEBUG] Kubernetes API Request Details:
2020-05-19T13:08:22.817+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: ---[ REQUEST ]---------------------------------------
2020-05-19T13:08:22.817+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: GET /api/v1/namespaces/service/serviceaccounts/service HTTP/1.1
2020-05-19T13:08:22.817+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: -----------------------------------------------------
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: 2020/05/19 13:08:22 [DEBUG] Kubernetes API Response Details:
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: ---[ RESPONSE ]--------------------------------------
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: {
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "kind": "ServiceAccount",
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "apiVersion": "v1",
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "metadata": {
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "name": "service",
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "namespace": "service",
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "creationTimestamp": "2020-05-14T12:21:28Z",
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  },
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "secrets": [
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   {
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:    "name": "service-token-kjpmt"
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   }
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  ]
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: }
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: -----------------------------------------------------
2020-05-19T13:08:22.955+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: 2020/05/19 13:08:22 [DEBUG] Kubernetes API Request Details:
2020-05-19T13:08:22.955+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: ---[ REQUEST ]---------------------------------------
2020-05-19T13:08:22.955+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: GET /api/v1/namespaces/service/secrets/service-token-kjpmt HTTP/1.1
2020-05-19T13:08:22.955+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: -----------------------------------------------------
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: 2020/05/19 13:08:22 [DEBUG] Kubernetes API Response Details:
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: ---[ RESPONSE ]--------------------------------------
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: {
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "kind": "Secret",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "apiVersion": "v1",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "metadata": {
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "name": "service-token-kjpmt",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "namespace": "service",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "selfLink": "/api/v1/namespaces/service/secrets/service-token-kjpmt",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "uid": "58e7c0f0-95dd-11ea-be2c-005056a1182f",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "resourceVersion": "85652125",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "creationTimestamp": "2020-05-14T12:20:50Z",
...
2020-05-19T13:08:22.961+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: 2020/05/19 13:08:22 [DEBUG] Skipping service-token-kjpmt as it existed before the service account
2020/05/19 13:08:22 [ERROR] <root>: eval: *terraform.EvalImportState, err: Failed to discover the default service account token: Unable to find any service accounts tokens which could have been the default one
2020/05/19 13:08:22 [ERROR] <root>: eval: *terraform.EvalSequence, err: Failed to discover the default service account token: Unable to find any service accounts tokens which could have been the default one

Expected Behavior

Resource should have been imported. The ServiceAccount references its own token by name, so why not import it ?

Actual Behavior

creationTimestamps are compared and because the Secret somehow got a creationTimestamp before the ServiceAccount, the import fails. If the timestamp has a later time it fails too... so if i delete the secret, which gets recreated automatically, it has a newer date and fails

Correct me if i'm wrong, but shouldnt it be enough to query the secret name from the service account and use that ? And the Secret also has annotations referencing the serviceaccount it belongs to

metadata:
  annotations:
    kubernetes.io/service-account.name: service
    kubernetes.io/service-account.uid: 7016a8d2-95dd-11ea-8c96-005056a1fd37

If i add another token to the serviceaccount, there is only an outgoing reference to the serviceaccount...

Any info welcome ;)

[tdmalone]

Workaround to import an existing ServiceAccount: (note: YMMV, manual state surgery is not a good idea, use with care, always have a backup)

1. Add configuration for a kubernetes_service_account in Terraform, but use a different name in the metadata (eg. your-secret-name-2)
2. Apply the configuration, creating the dummy ServiceAccount
3. Manually delete the dummy ServiceAccount, eg. using kubectl delete sa your-secret-name-2
4. Get the default secret name from your real ServiceAccount by running eg. kubectl describe sa your-secret-name, noting down the value for Tokens: something like your-secret-name-abc12
5. Run terraform state pull > temp.state and open temp.state in your code editor
6. Look for each instance of your-secret-name-2 and replace it with your-secret-name (there should be about 4 - in the default_secret_name, id, name and self_link)
7. In the same resource, change the value of default_secret_name to match that you noted down above (eg. your-secret-name-abc12)
8. Near the very top of your state file, increment the numerical value for serial
9. Save the file, then run terraform state push temp.state
10. If you haven't already, change the name of your kubernetes_service_account back to the name it's meant to be (i.e. your-secret-name rather than your-secret-name-2
11. Run terraform plan - if everything's gone well Terraform should now see your pre-existing ServiceAccount and report no changes to it

[erks]

Adding more evidence that the current import logic seems too fragile.

In my case, the creationTimestamp of the Secret is 7 seconds after that of the Service Account, causing the import to fail with this error:

2021-01-10T07:09:54.893Z [DEBUG] plugin.terraform-provider-kubernetes_v1.13.3_x4: 2021/01/10 07:09:54 [DEBUG] Skipping aws-node-token-zsgw8 as it wasn't created at the same time as the service account

[robcoward]

I've also just hit this bug. Not sure how, but in this case the token appears to have been refreshed on the serviceAccount, as the timestamp on the service account is "2021-01-11T17:27:28Z" but the timestamp on the secret is "2021-06-25T12:04:59Z" and it is being skipped with the error

2021-06-29T09:51:44.199Z [INFO]  plugin.terraform-provider-kubernetes_v2.3.2_x5: 2021/06/29 09:51:44 [DEBUG] Skipping cluster-autoscaler-token-zp4c9 as it wasn't created at the same time as the service account: timestamp=2021-06-29T09:51:44.199Z

[github-actions[bot]]

Marking this issue as stale due to inactivity. If this issue receives no comments in the next 30 days it will automatically be closed. If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. This helps our maintainers find and focus on the active issues. Maintainers may also remove the stale label at their discretion. Thank you!

[tdmalone]

unstale

[github-actions[bot]]

Marking this issue as stale due to inactivity. If this issue receives no comments in the next 30 days it will automatically be closed. If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. This helps our maintainers find and focus on the active issues. Maintainers may also remove the stale label at their discretion. Thank you!

[rossmobi]

Not stale.