caddy icon indicating copy to clipboard operation
caddy copied to clipboard

Can we make reverse proxy response available in templates

Open byoungdale opened this issue 4 years ago • 9 comments

This is in response to my question here on the community forum.

Can we add a feature where the response to a reverse proxy can be made available to or passed as argument(s) to templates?

:9000

reverse_proxy 127.0.0.1:3000 {
              handle_response {
                              templates
                              try_files posts.html
                              file_server
              }
}

And if the response from the 127.0.0.1:3000 application is:

[
  {
    "id": 1,
    "title": "Post 1"
  },
  {
    "id": 2,
    "title": "Post 2"
  },
  {
    "id": 3,
    "title": "Post 4"
  },
]

Being able to do something like this:

<!DOCTYPE html>
<html>
	<body>
		<main>
			<div class="post-container">
                                 {{range $field, $val := .Res.Body}}
                                         	<p id={{$val.id}}>{{$val.title}}</p>
                                  {{end}}
			</div>
		</main>
	</body>
</html>

byoungdale avatar Nov 15 '21 00:11 byoungdale

Hmm. I think you'd be better off by responding with HTML that has JS fetch() calls to get the JSON, and render that however you need with JS.

Or, write a separate backend endpoint that renders HTML from the the JSON with an actual programming language.

There's a bunch of pieces in play here that make this tricky to implement, and I'm afraid it'll add undue complexity to the system:

  • Response may be huge, making it available in templates would mean reading it all and buffering it into memory
  • Making the data available in templates could require some complex pipework
  • The response may not necessarily be JSON, decoding it to make it available in an map, and probably not handle other kinds of responses seems weird

francislavoie avatar Nov 15 '21 01:11 francislavoie

Yeah, this'll definitely require buffering the response. Maybe if it was explicitly enabled in config (up to a max size, I dunno?) that would be one easy way to do it. For example, "Enabling this option always reads the response from the backend up to a certain size and then sets these placeholders/variables, etc."

mholt avatar Nov 15 '21 18:11 mholt

FYI, this should be closer to being possible to implement now that we have a copy_response handler in handle_response routes (as of v2.5.0-beta.1).

Parsing the response is a whole other thing though, and likely best left to plugins once #4568 is done as well.

francislavoie avatar Mar 26 '22 01:03 francislavoie

it seems working under v2.5.1, but how to turn it off?

mindon avatar Jul 01 '22 04:07 mindon

@mindon you'll need to elaborate. What's your config? What behaviour are you seeing? What does it look like when you make a request with curl -v?

francislavoie avatar Jul 03 '22 21:07 francislavoie

@mindon you'll need to elaborate. What's your config? What behaviour are you seeing? What does it look like when you make a request with curl -v?

curl -v works fine, the issue pattern is encode zstd gzip → reverse_proxy → templates

remove encode zstd gzip from target server or the templates from current server deliminates the 500 error

https://github.com/caddyserver/caddy/issues/4819

here's a simple config

{
  debug
	auto_https off
}

:8081 {
  encode zstd gzip
  root * ./hello
  route /world/* {
    uri strip_prefix /world
    header X-Real-IP {http.request.remote}
    reverse_proxy 127.0.0.1:8080
  }
  file_server
  templates
}

:8080 {
  encode zstd gzip
  root * ./world
  file_server
  templates
}

mindon avatar Jul 04 '22 10:07 mindon

@francislavoie a simple case is proxy target output something like "{{}}", then caddy app with templates will surely 500 error

mindon avatar Jul 06 '22 02:07 mindon

That seems off-topic from what this issue is about @mindon.

The feature request here is about being able to write the proxy response from within a template, with handle_response. Your config doesn't do that.

francislavoie avatar Jul 06 '22 03:07 francislavoie

@byoungdale Please see #4871 where I've implemented this functionality. It's a little rough around the edges still, but it should work.

If you want to parse JSON, simply use the fromJSON or mustFromJSON function: https://masterminds.github.io/sprig/defaults.html#fromjson-mustfromjson

mholt avatar Jul 07 '22 19:07 mholt