api: Support for batch config modifications

[francislavoie]

The Caddy API in v2 is very powerful.

One area that I found it lacking is in being able to make batch modifications in one atomic operation. Currently, the only possible way to do this would be to GET the whole config (or at least a superset that contains all the sections you wish to modify), make the JSON modifications offline, then POST the entire set back to the API.

An approach I thought of that could improve this is by having a JSON-RPC endpoint available in the API. This would allow for doing batch operations to the JSON config atomically, i.e. Caddy would only reload one time, after the entire batch is executed.

For example, here's a possible JSON-RPC request payload to modify two different servers' listen addresses at once:

$ curl -X POST \
    -H "Content-Type: application/json" \
	-d '
	[
	  {
	    "json-rpc": "2.0",
	    "method": "PUT",
	    "params": {
	      "path": "/config/apps/http/servers/srv0/listen/0",
	      "data": ":8080"
	    }
	  },
	  {
	    "json-rpc": "2.0",
	    "method": "PUT",
	    "params": {
	      "path": "/config/apps/http/servers/srv1/listen/0",
	      "data": ":8081"
	    }
	  }
	]' \
    "http://localhost:2019/json-rpc"

This would essentially be the same as doing the following, except that the following would trigger two config reloads because those two operations are not atomic:

$ curl -X PUT \
	-H "Content-Type: application/json" \
	-d '":8080"' \
	"http://localhost:2019/config/apps/http/servers/srv0/listen/0"

$ curl -X PUT \
	-H "Content-Type: application/json" \
	-d '":8081"' \
	"http://localhost:2019/config/apps/http/servers/srv1/listen/0"

This issue is mainly to guage interest to see whether anyone else would actually need this sort of feature -- I don't immediately need it myself, but I spotted the lack in the current API so I figured I'd type up a potential solution.

[sospartan]

Caddy would only reload one time, after the entire batch is executed.

One feature that bother me is why Caddy restart everything when config changed, not the app(s) the changes do affected?

[mholt]

@sospartan Because a config change in one app can affect others that may rely on that config. (Apps can refer to other apps.) That's not a problem though; "restarting everything" is actually a lot less work than you think, and config changes are designed to be very quick and smooth in Caddy 2. In this video I go into depth on how it works (turn volume up, their audio was crummy): https://www.youtube.com/watch?v=EhJO8giOqQs&list=PLVmzv2RTGVDGbPou-wLIDP2QEYKj0Xi-i&index=32

[sospartan]

@mholt thanks for the video, this talk is very helpful.

[mholt]

I think this might be better handled by an admin module, ala #5032 -- which is not currently a thing, but it's on my list!

Also note for everyone following this issue, that Caddy's admin API now supports Etags to ensure consistent, concurrent changes to the config. It's a different model of synchronization, but it is an option that you can use right now if you want to.

I'm tagging this issue as "plugin" now, since it seems to me that will be the most likely way forward, and it will probably be a plugin that will live outside this repo to start with. We can always adopt it in later if it's very popular and has great general utility. But given that it will likely be a third-party feature, but relies on #5032, I'll close this issue.

Feel free to continue discussion here about this functionality! I'm just closing it because it won't really be actionable by this repo. My focus will be on getting #5032 done sometime after Caddy 2.6 is released.