Import existing certificates to prevent rate limits

[designermonkey]

I am trying to create a working terraform codebase which has had errors here and there during development. To help in this I have been able to import existing resources to the state using terraform import command.

I have just hit a rate limit with the letsencrypt API that I am now blocked from developing on this codebase for 24hrs. I went to the docs to see how to import certificates like with other resource providers, and there is no mention of that functionality.

Would it be possible to add that ability to this provider please?

[bukowa]

I run make cleanup and then make apply and it works for my case, basically controlling targets via Makefile works wonders for me. The point is to delete other resources with --target but to keep acme stuff.

apply:
	terraform apply --auto-approve --target=acme_registration.reg
	terraform apply --auto-approve --target=digitalocean_droplet.nginx_proxy
	terraform apply --auto-approve

cleanup:
	terraform destroy --auto-approve --target=digitalocean_record.domain
	terraform destroy --auto-approve --target=digitalocean_droplet.nginx_proxy

[vancluever]

@designermonkey @bukowa I'll mull this and see how I can make it better. In addition to targeted destroys (and probably even better than that), you can set up your certificates in another Terraform workspace and just source the certificates from state with terraform_remote_state.

Also to try and keep yourself from being locked out of production, you can also use the staging server, which is specifically designed for pre-deployment development.

Thanks!

[designermonkey]

Thanks for the input, I realised I had the prod server in use and did a massive facepalm.

[atrifyllis]

It would be great to reuse certificate e.g. reusing account and not regenerating if certificate already exists (no idea if this is possible). Otherwise how do you deal with production certificates (some kind of best practice). If using the tf remote state, what do you retrieve from it and how do you use it?

[vancluever]

@atrifyllis:

• Account re-use should already be happening, you should not be re-creating acme_registration every time a certificate is renewed.
• Private key may be re-generated but this can be controlled by just using an external CSR if you don't want it to be.
• A renewed certificate is always a new certificate by definition as it will contain entirely new information with a new cryptographic signature (along with a new validity notbefore/notafter range).

So, technically, certificate "re-use" is not possible.

Implementation of a rolling-update scenario that best suits your needs is outside our scope. Optimally you should be renewing with adequate time to deal with deployment issues and backing up and/or versioning your Terraform state using a solution that allows you to go back to previous state if for some reason you lose the certificate and private key mid-deploy.

Thanks!

[atrifyllis]

What I meant by reuse was after a destroy not after a renewal. I would expect some kind of account reuse in that case. Why would you recreate account and regenerate cert? You might argue that production is not recreated so often and you would not hit rate limits. But actually using self signed certs from staging causes issues in all envs (e.g. many javascript auth libraries do not allow calling a backend that is behind a self signed cert)

[vancluever]

@atrifyllis nothing in the renewal process should not force a new resource (that's why we added updating in the first place through CustomizeDiff several years ago). Can you give me an example of what you mean?

As far as staging goes, I can't really offer anything here, sorry. Staging is specifically the environment Let's Encrypt reserves for folks that need to test their configurations and is designed for that purpose and has better rate limits than production. If you need to test, you can probably check the javascript libraries you use for an "insecure" flag that allows you to skip validation, or add the staging CA certificate in your environment's root store.

[atrifyllis]

Thanks for all the feedback! You ask for an example, the use case is pretty clear I think. I want to reuse a certificate somehow and not regenerate it every t destroy then apply.

[vancluever]

@atrifyllis you should not be running terraform destroy (or other commands that will similarly destroy a resource) every time you want to renew a certificate. By Terraform convention, destroy is an explicit command to Terraform to tear down infrastructure. The ACME provider interprets that as revoking the certificate.

You should rather simply let the resource renew naturally through the time controlled by the min_days_remaining attribute. You can also force a renew by adjusting this value to a value higher than the time remaining on the certificate.

revoke_certificate_on_destroy was also added relatively recently to allow scenarios where you may not want a certificate to be explicitly revoked on a destroy, you could try setting that to false.

Hope that helps!