website icon indicating copy to clipboard operation
website copied to clipboard

Examples to add to the docs

Open mholt opened this issue 1 year ago • 23 comments

We are planning to add a large repository of examples to the docs.

However, the breadth of use cases is nearly infinite and there are only a few of us maintainers, so we need your help by posting your configs for your use cases here.

Please post your examples here. Doesn't matter how niche; let us decide that. If you think it's useful, or something you wish you had when you were starting, share it!

We are accepting both JSON and Caddyfile configurations. Note that we may revise examples when posting them to our site. Feel free to reply here with your examples to contribute.

Instructions

  • Describe the use case (could be a backend app name, a specific config task, an integration, something to do with auto HTTPS/TLS, a clever solution, etc.)
  • Include at least the minimal config needed. You may include variations of your example, or additional configuration that may be "recommended," but please annotate them as optional and explain what they do in comments.
  • Comments are useful, but some things are obvious and don't need comments.
  • Link to any source if you got the example from a web page (e.g. if a third-party app shows how to integrate with Caddy on their docs, please link to that page).

I would love to see 100+ examples posted here from all of you.

Thank you!

Here's an example :wink:

WordPress

example.com

root * /var/www/wordpress
php_fastcgi unix//run/php/php-version-fpm.sock
file_server

# block access to sensitive paths (optional, recommended)
@blocked path /xmlrpc.php *.sql /wp-content/uploads/*.php
rewrite @blocked /index.php

# enable compression (optional)
encode gzip

mholt avatar Jun 04 '23 17:06 mholt

@mholt happy to help. working / maintained examples are really needed.

How about making a list of types of examples that you feel are appropriate, so then people can co-ordinate around that ?

gedw99 avatar Jun 25 '23 13:06 gedw99

Basically, just post your Caddy configs you're already using and we'll take it from there. It would be nice if you comment parts that aren't obvious, or which are optional, that could be helpful.

mholt avatar Jun 25 '23 16:06 mholt

I have a request for a config example. My general reverse proxy config looks like this:

example.com {
  reverse_proxy :5001
  encode gzip
}

I used to have a directive in there to serve a simple maintenance.html page whenever the app on port 5001 was offline — because I was deploying an update for example. But the way to do this in Caddy changed and I can't figure out how to do this today.

karimfromjordan avatar Jun 29 '23 08:06 karimfromjordan

@karimfromjordan you're looking for https://caddyserver.com/docs/caddyfile/directives/handle_errors. The reverse_proxy will trigger an error if the upstream can't be connected to, and you can handle it with handle_errors to do something else instead.

francislavoie avatar Jun 29 '23 09:06 francislavoie

We make heavy use of snippets and ENV's for our TLS on-demand setup

{
    order rate_limit before basicauth

    on_demand_tls {
        ask      {$CADDY_ASK_URL}
        interval {$CADDY_ASK_INTERVAL}
        burst    {$CADDY_ASK_BURST}
    }
    servers {
        metrics
        trusted_proxies static 0.0.0.0/0
        client_ip_headers CF-Connecting-IP X-Forwarded-For
    }
}

(common) {
    rate_limit {
        distributed
        zone ip_rate {
            key    {client_ip}
            events {$CADDY_RATE_EVENTS:10}
            window {$CADDY_RATE_WINDOW:1s}
        }
    }

    @online {
        expression "{env.CADDY_MAINTENANCE_ENABLED} == \"false\""
    }

    @offline {
        expression "{env.CADDY_MAINTENANCE_ENABLED} == \"true\""
    }

    @whitelist {
        path {$CADDY_PATH_WHITELIST:*}
    }
}

(tls_ondemand) {
    tls {
        on_demand
    }
}

(tls_cloudflare) {
    tls {
        dns cloudflare {$CLOUDFLARE_API_TOKEN}
    }
}

(online_handle) {
    handle @online {
        reverse_proxy @whitelist {$CADDY_BACKEND_HOST}:{$CADDY_BACKEND_PORT}
    }
}

(offline_handle) {
    handle @offline {
        reverse_proxy {$CADDY_MAINTENANCE_HOST:https://maintenance.com} {
            header_up Host {$CADDY_MAINTENANCE_HEADER_HOST:maintenance.com}
        }
    }
}

(fallback_handle) {
    handle {
        respond "" 404
    }
}

:8080 {
    respond /healthz "200 ok"
}

import extra/*

:443 {
    import common
    import tls_ondemand

    import online_handle
    import offline_handle

    import fallback_handle
}

test.com {
    import common
    import tls_cloudflare

    import online_handle
    import offline_handle

    import fallback_handle
}

robgordon89 avatar Jun 29 '23 17:06 robgordon89

This is what I have running at the moment. I have 2 caddy servers to achieve high availability. Their only function is to get SSL certificates and do reverse proxying. Both Caddy servers run in docker and share the same data folder. They both start with a basic config.json telling Caddy to load the actual config using HTTP.

Initial config present on both Caddy servers:

{
    "admin": {
        "config": {
            "load_delay": "30s",
            "load": {
                "module": "http",
                "url": "http://10.10.2.157:30050/config.json"
	    }
        }
    }
}

Config that is centrally hosted and gets loaded through HTTP:

{
    "admin": {
        "config": {
            "load_delay": "30s",
            "load": {
                "module": "http",
                "url": "http://10.10.2.157:30050/config.json"
	    }
        }
    },
    "apps": {
        "http": {
            "servers": {
                "reverse-proxy": {
                    "listen": [":443"],
                    "routes": [
                        {
                            "match": [
                                {
                                    "host": ["subdomain1.example.com"]
                                }
                            ],
                            "handle": [
                                {
                                    "handler": "reverse_proxy",
                                    "upstreams": [
                                        {
                                            "dial": "tcp/10.10.2.157:32001"
                                        }
                                    ]
                                }
                            ]
                        },
                        {
                            "match": [
                                {
                                    "host": ["subdomain2.example.com"]
                                }
                            ],
                            "handle": [
                                {
                                    "handler": "reverse_proxy",
                                    "upstreams": [
                                        {
                                            "dial": "tcp/10.10.2.157:32100"
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            }
        }
    }
}

MattsBos avatar Jun 30 '23 21:06 MattsBos

Related: https://github.com/search?q=%28path%3A**%2FCaddyfile+OR+path%3A**%2F*.caddy%29&type=code

Would be nice to scrape all of these (~7k apparently, probably a fraction of that after dedepulicating), then put a simple but blazing fast search on top of them (could even be client side only). The use case: often you have an idea or problem you need to solve, and the fastest way to validate that and see how it's possible is to search a corpus of config others wrote. Also, often the best way to understand a directive or config parameter is to see the different ways it's used in the real world.

(tangent: I tried to add the created:>2020 stars:>100 qualifiers to the search but GH didn't show anything then.)

mustafa0x avatar Jul 10 '23 04:07 mustafa0x

feels like we need a high level categorisation from easy to hard to classify the config examples.

and then a examples repo where the examples are exercised. The endpoints just need to respond with fake data - so just a simple main.go for each endpoint.

Also would suggest using overmind to run each example. Its just a procFile for each example folder. https://github.com/DarthSim/overmind

In the end, this would make it so much easier to use caddy and have someone to go to see when things screw up in your config.

gedw99 avatar Jul 10 '23 10:07 gedw99

I would suggest

example.com {
  handle_path /foo {
    reverse_proxy localhost:1234
  }
}

as an example for consolidating microservices under domains or similar.

randomairborne avatar Jul 10 '23 19:07 randomairborne

@mustafa0x Thanks for that search, I dunno how useful a pile of 7K of them will be, but maybe I can scan those and look for some common patterns that stick out to me!

@gedw99 Evaluating examples is a whole 'nother dealio; maybe we can have a Caddy sandbox/playground, but the new site won't initially have one because of priorities.

@randomairborne Thanks! I will have to make a note with that example that most backend apps don't like being proxied to in a "subfolder" unless they're explicitly configured to do so.

mholt avatar Jul 10 '23 19:07 mholt

I'm happy to see this issue is getting some love. It's one thing having helpful users and developers in the community site, but standalone resources and examples would have been brilliant for me.

I have a mix of elegant and horrible examples… so you can have one of each, for now.

CORS

This is based on Setup CORS in Caddy 2 and Caddy Server, CORS, and Preflight Requests - AUXNET but changes (fixes?) a few things.

  1. Adds Vary: Origin to prevent caching.
  2. Adds defer to prevent duplicate headers.
  3. Adds {args.0} to the names to enable multiple cors snippets.
  4. Customizes the Access-Control-Expose-Headers.
  5. Allows credentials to be sent with cross origin requests.
  6. [edit] Can be imported multiple times to support multiple Origins (more precise matching on pre-flight).

I'm not 💯 it's doing all the right things; check for your own use case.

(cors) {
  @cors_preflight{args.0} {
    method OPTIONS
    header Origin {args.0}
  }
  @cors{args.0} header Origin {args.0}

  handle @cors_preflight{args.0} {
    header {
      Access-Control-Allow-Origin "{args.0}"
      Access-Control-Allow-Credentials true
      Access-Control-Allow-Headers "Authorization, Cache-Control, Content-Type"
      Access-Control-Allow-Methods "GET, POST, PUT, PATCH, DELETE"
      Access-Control-Max-Age "3600"
      Vary Origin
      defer
    }
    respond "" 204
  }

  handle @cors{args.0} {
    header {
      Access-Control-Allow-Origin "{args.0}"
      Access-Control-Allow-Credentials true
      Access-Control-Expose-Headers "{args.1}"
      Vary Origin
      defer
    }
  }
}

Use as follows:

import cors https://{$HOST} "ETag"

Rewriting query parameters

This is a bit of a bruiser… and I think there's a race condition, as the iteration ordering of go maps is not defined (but is usually alphabetical…) — if I can reproduce it, I'll raise an issue.

This snippet pulls out a query parameter that is set to true, and builds a list of ,-separated parameters instead.

(query_select) {
  @query_{args.0} query {args.0}=true
  vars @query_{args.0} {
    select "{vars.select}{vars.sep}{args.0}"
    sep ","
  }
}

Use as follows.

https://{$HOST} {
  import query_select first
  import query_select second
 
  rewrite * /?new_query={vars.select} 
  …
}

This then converts https://{$HOST}/?first=true&second=true to https://{$HOST}/?new_query=first,second for subsequent use.

dbaynard avatar Jul 10 '23 20:07 dbaynard

@mholt this one is really specific to things like APIs that don't directly interact with the user, imo

randomairborne avatar Jul 11 '23 02:07 randomairborne

@dbaynard Thanks, there's some good content there.

@randomairborne Which one, exactly?

mholt avatar Jul 11 '23 21:07 mholt

@mholt https://github.com/caddyserver/website/issues/324#issuecomment-1629608267 is specifically for a lot of API servers, and will utterly break a lot of more traditional templated websites.

randomairborne avatar Jul 11 '23 21:07 randomairborne

Yes, true -- as I said above, most websites don't like that kind of proxying, so we'd have to make a note of it. I like the distinction between templated sites and "API servers". Makes a lot of sense.

mholt avatar Jul 11 '23 23:07 mholt

CORS

This is based on Setup CORS in Caddy 2 and Caddy Server, CORS, and Preflight Requests - AUXNET but changes (fixes?) a few things.

  1. Adds Vary: Origin to prevent caching.
  2. Adds defer to prevent duplicate headers.
  3. Adds {args.0} to the names to enable multiple cors snippets.
  4. Customizes the Access-Control-Expose-Headers.
  5. Allows credentials to be sent with cross origin requests.
  6. [edit] Can be imported multiple times to support multiple Origins (more precise matching on pre-flight).

I'm not 💯 it's doing all the right things; check for your own use case.

I found and fixed an issue, with this (the issue is in the sources I referenced). When imported multiple times, only the origin from the first was allowed. Now only the pre-flight with the correct origin matches.

(cors) {
  @cors_preflight{args.0} {
    method OPTIONS
    header Origin {args.0}
  }
  @cors{args.0} header Origin {args.0}

  handle @cors_preflight{args.0} {
    header {
      Access-Control-Allow-Origin "{args.0}"
      Access-Control-Allow-Credentials true
      Access-Control-Allow-Headers "Authorization, Cache-Control, Content-Type"
      Access-Control-Allow-Methods "GET, POST, PUT, PATCH, DELETE"
      Access-Control-Max-Age "3600"
      Vary Origin
      defer
    }
    respond "" 204
  }

  handle @cors{args.0} {
    header {
      Access-Control-Allow-Origin "{args.0}"
      Access-Control-Allow-Credentials true
      Access-Control-Expose-Headers "{args.1}"
      Vary Origin
      defer
    }
  }
}

dbaynard avatar Jul 27 '23 10:07 dbaynard

@robgordon89

We make heavy use of snippets and ENV's for our TLS on-demand setup

How are you passing the ENV to Caddy (--envfile?) and how do you do reloads when you want to change for example CADDY_MAINTENANCE_ENABLED from false to true to take effect?

I definitely like the approach with environment variables, but caddy reload does not accept the --envfile flag.

steffenbusch avatar Jul 30 '23 11:07 steffenbusch

but caddy reload does not accept the --envfile flag.

We could probably change that. I don't think it would be able to reset the environment but we could at least update any prior/existing values and add new ones.

mholt avatar Jul 31 '23 20:07 mholt

We could probably change that. I don't think it would be able to reset the environment but we could at least update any prior/existing values and add new ones.

Hi Matt, I think that would be very helpful to have --envfile in the reload command available. Thank you for considering this.

I'm still starting with Caddy Server configuration and still doing lots of tests and trials regarding my configuration. I have to adopt a lot of logic and features from Caucho Resin to Caddy. An ability to have Caddy reload the Caddyfile configuration with consideration of values from envfile would be great. Honestly, I have not yet understood (from the docs) what's the difference between using {$CADDY_MAINTENANCE_ENABLED} and {env.CADDY_MAINTENANCE_ENABLED} in Caddyfile when it comes to environment variables.

steffenbusch avatar Aug 01 '23 12:08 steffenbusch

Getting off topic here, but I don't see how reload --envfile would work. A reload involves pushing the new config via Caddy's API, and I don't see how we can also push the envfile contents at the same time, unless we encode it as HTTP headers or something which is strange.

Regarding the difference between {$VAR} and {env.VAR}, the former is a textual replacement in the Caddyfile before it is parsed (so it can be placed anywhere and expand into many config tokens) and the latter is replaced at runtime, and can only be used in config locations where the replacer is run (in practice this is most places, but with many limitations). For example: for secrets, the latter is preferred because it will hide it from the autosave.json file since it's a static value, but for a domain name (site address) you need to use the former because it needs to be known up-front and can't be replaced every time it's needed at runtime.

I'll try to clean up the explanation on https://caddyserver.com/docs/caddyfile/concepts#environment-variables but it's essentially saying the same thing.

francislavoie avatar Aug 01 '23 12:08 francislavoie

The second paragraph is an awesome explanation. That would be very beneficial for the documentation.

steffenbusch avatar Aug 01 '23 12:08 steffenbusch

At a meta level, I'm happy to see you (maintainers) going through this process. I came to caddy from tailscale, and there are many things I like about it, but it has been difficult to figure out how to use its functionality, in particular for what must be common use cases. So I'm happy to help contribute, too.

dbaynard avatar Aug 01 '23 13:08 dbaynard

I've hit a situation that the CORS block above doesn't handle: if the Origin should be a regex. There's a straightforward modification to handle this situation, and I've created a variable to have the actual matched origin.

However, in doing so, I found bugs in my configuration that took me a long time to resolve. As part of it, I learned how to interpret the actual caddy json structure. Turns out I had to wrap the import cors lines together in a route block, and everything else in another route block, as the two are not mutually exclusive (and also the cors matchers have to run first to extract the actual origin, where all I know is the regex).

I did know about Similar directives — handle (Caddyfile directive) — Caddy Documentation but didn't clock that the reason caddy was returning a 200 where I had intended a 418 is because the CORS handlers and the actual routing logic were mutually exclusive.

The differences between handle and route seem pretty crucial to understanding the control flow through caddy, and so I think it would be helpful to show an example like this, to give it more prominence. I know they are introduced in Directives — Caddyfile Concepts — Caddy Documentation; I think it's easy to miss.

(I'm not particularly obsessed with CORS configuration, it's just that's where my attention is needed…)

dbaynard avatar Aug 09 '23 23:08 dbaynard