nebari-dev
[BUG] - Traefik rate limited by Let's Encrypt after several pod restarts
Describe the bug
Traefik documentation warns to persist the acme.json certificate storage to avoid being rate limited by Let's Encrypt, but Nebari's deployment isn't configuring that. If the traefik pod is restarted enough times within a given week without this setup, Let's Encrypt will rate limit the certificate issue request and cause the site to default back to self-signed.
This has happened to me many, many times when I'm developing or testing changes to the platform or dealing with conda storage related node crashes, usually requiring a workaround involving installation of cert-manager and switching to zerossl or another cert issuing service until the Let's Encrypt backoff period elapses.
Expected behavior
traefik pod restarts should not result in new certificates being issued when configured for lets-encrypt.
Options:
- Configure a persistent volume mount for the acme.json storage path.
- Replace Traefik's native Let's Encrypt integration with cert-manager for more issuer options and improved scalability.
OS and architecture in which you are running Nebari
Linux, x64
How to Reproduce the problem?
- Configure the Nebari certificate with lets-encrypt
certificate:
type: lets-encrypt
acme_email: [email protected]
acme_server: https://acme-v02.api.letsencrypt.org/directory
- Delete the traefik pod 5 or more times in 1 week.
Command output
No response
Versions and dependencies used.
Nebari 2023.11.1
Compute environment
AWS
Integrations
No response
Anything else?
No response
I am working on this ticket, Please find the gitlab issue as below. https://gitlab.jatic.net/jatic/team-metrostar/t-e-platform/-/issues/347
