caddyserver
Disabling the built-in Tailscale behavior
Issue Details
From the docs:
Special cases:
Domains ending in .ts.net will not be managed by Caddy. Instead, Caddy will automatically attempt to get these certificates at handshake-time from the locally-running Tailscale instance. This requires that HTTPS is enabled in your Tailscale account and the Caddy process must either be running as root, or you must configure tailscaled to give your Caddy user permission to fetch certificates.
This does not mention how to disable this behavior. Attempting to turn off auto_https does not appear to work.
There should probably also be a discussion around whether or not this special case default makes sense- it means that configurations that work perfectly well on one domain unexpectedly break on another, which is surprising and quite frustrating.
Assistance Disclosure
AI not used
If AI was used, describe the extent to which it was used.
No response
What are you actually trying to do? If you just want to serve HTTP, just specify http://<your-domain>.ts.net
Show your actual config, show the actual problem you're having, with logs etc.
Hm, yeah I'm curious what issues you're having. Never had a request for this in the ... 4 years? we've had this out there.
To clarify, this issue is about the documentation (and possibly the implementation- its unclear) regardless of the specifics of my use case. The issue is that there is a special carve-out for domains hosted by tailscale, but no documentation regarding how to disable that special behavior.
In my particular case, there is an internal tool that requires https and I'm trying to use caddy's tls on_demand behavior.
I don't know if others are in the same boat, but certainly this is not the first time I've hit this (and I know my coworkers have also run into this) but it is the first time I've decided to file an issue. Mostly because I cannot seem to figure out how I got around it last time- possibly just made it so the domain is only accessible if you leave of the ts suffix.
I'm confused. If you're using Tailscale, why not just use Tailscale's HTTPS? You can't issue a publicly trusted certificate with a domain you don't control (and you don't control DNS for .ts.net so you can't anyway), so on_demand can never work for those domains.
This sounds very XY problem, could you please describe what you're actually trying to do?
I considered preemptively stating that I do not believe this is an XY problem…
Again, even if you feel my particular approach is invalid, this is still a gap in the documentation.
In this particular case, as stated, this is for an internal tool. I don’t need a publicly trusted cert. The domain name may be different depending on deployment so on demand is useful. At some point in the future we will adopt a more formal cert management process, but it would be really convenient to use caddy to just generate the certs for now.
Please note that the entire above explanation applies only to my particular circumstances and has no bearing on the state of the documentation, which is the concern of this issue.
The domain name may be different depending on deployment so on demand is useful.
That's not what on-demand is for. If the domain is known at config time, you should fill it in with an environment variable or whatever config generation you want. On-demand is for when you're trying to serve customers for which you don't know their domains ahead of time and would need to be issued dynamically.
I still don't understand what you're trying to actually do without a config example or logs, so there's nothing actionable here. We can't make a documentation change without proper context. We don't know where you're coming from with this so we can put ourselves in your shoes to fix the documentation such that if we were you we'd understand whatever you think is unclear.
Apologies for the confusion; I will try to clarify.
If I have a given Caddy config that I deploy on a domain that does not end in .ts.net I get behavior A.
If I have a given Caddy config that I deploy on a domain that does end in .ts.net I get behavior B.
The crux of this issue is: how can I ensure I get behavior A regardless of deployment location?
I do not believe the particulars of behavior A are critical- the point is that the tailscale carve out results in different behaviors in different environments, and the question at hand is how to make caddy behave consistently across domains.
An additional example-
Let's say I have three environments:
- Application in docker on domain
app-east.company.internal - Application in docker on domain
app-west.company.internal - Application in docker on domain
app-north.company.ts.net
All applications have identical caddy configs that use https.
In environments 1 and 2, the deployment behavior will be identical. In environment 3, the deployment behavior will be entirely different due to the special case of ts.net. I would like to understand how I can disable the special case for ts.net such that those environments behave identically.
I understand that the answer to this question might be "There is no way to disable that special case, such that the environments will behave consistently."
The answer might also be "There is no way explicitly target the special case, but certain configurations will not encounter this behavior." It would be helpful to understand (and document) what those configurations are. It is also possible that the internals are complex enough that nobody knows, but that also seems like a problem.
I appreciate the help here.
But why are you using .ts.net if you also don't want Tailscale's HTTPS certs? And if you don't want Tailscale's HTTPS certs, what certs are you expecting to use? Caddy can't serve HTTPS without valid certs. If you want Caddy's generated certs from its CA, just specify tls internal in your site block and it will do that.
Thank you!
I haven't actually tested this yet but that makes sense to me. The implication is that absent any other config Caddy will use the TLS internal behavior, but that gets "overridden" if you're on a .ts.net domain. So if you want to keep that behavior even when on a .ts.net domain, you just need to explicitly call out its use.
I think it would be really useful to put this in the docs where the bit about tailscale is mentioned.
Cool, glad we finally aligned on the problem.
If you specifically use .internal then it activates tls internal implicitly as mentioned here https://caddyserver.com/docs/automatic-https#hostname-requirements, so you're already using implicit behaviour for that case. So using tls internal explicitly if it's what you want in the other case does make sense.
Some clarifications can probably be made to the Automatic HTTPS page to clarify some of those details.
Yes, maybe in the Activation section, something like:
Caddy attempts to automatically determine the appropriate certificate issuer based on environment (for example, using the
internalissuer for domains ending in.internal). To override this behavior, explicitly configure the tls issuer.
And then in the Special Note for Tailscale:
Domains ending in .ts.net will not be managed by Caddy. Instead, Caddy will use a Tailscale-specific issuer, which automatically attempts to get these certificates at handshake-time from the locally-running Tailscale instance. This requires that HTTPS is enabled in your Tailscale account and the Caddy process must either be running as root, or you must configure tailscaled to give your Caddy user permission to fetch certificates. To change this behavior, explicitly configure an
issuer.
Assuming the above correct, I think it would really help the mental model for users who are unfamiliar with what the "magic" of Caddy actually does.
I had a similar situation.
The issue I ran into was that I wanted to have a single FQDN that I could use to access my server. When outside of my LAN, I can turn on Tailscale and use the FQDN provided my MagicDNS to access it through a caddy-tailscale docker instance that is working flawlessly.
However, when I am on my LAN, I would like to override my local DNS to return the IP address of the caddy server directly, so it doesn't need to go through Tailscale unnecessarily. So I tried to set up a second, normal caddy docker instance on a separate docker network that would respond only on the internal network. However I can't force caddy to use an internal tls for the ts.net domain, it just fails and tries to connect to the non-existent /var/run/tailscale/tailscaled.sock.
Is there a better way to accomplish this?
