haproxy
DOC - Clarify that httpclient directives apply to Lua only
Detailed Description of the Problem
To avoid confusion, could the following directives be updated in the configuration guide to indicate that they apply to Lua scripts? In one instance, a customer believed the "resolvers" directives applied to DNS resolution in backends.
httpclient.resolvers.disabledhttpclient.resolvers.idhttpclient.resolvers.preferhttpclient.retrieshttpclient.ssl.ca-filehttpclient.ssl.verifyhttpclient.timeout.connect
Expected Behavior
Doc clarifies the context of "httpclient".
Steps to Reproduce the Behavior
Documenation issue, no steps
Do you have any idea what may have caused this?
No response
Do you have an idea how to solve the issue?
No response
What is your configuration?
n/a
Output of haproxy -vv
HAProxy version 2.8.0-1.0.0-310.418 2023/12/14 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2028.
Known bugs: https://www.haproxy.com/documentation/hapee/2-8r1/onepage/changelog/#2.8.0
Running on: Linux 5.10.0-22-amd64 #1 SMP Debian 5.10.178-3 (2023-04-22) x86_64
Last Outputs and Backtraces
No response
Additional Information
No response
Hi Nick,
I'm not very clear about which ones are not clear, maybe this one, which seems to be the only one not explicitly mentioning "httpclient" ?
httpclient.resolvers.prefer <ipv4|ipv6>
This option allows to chose which family of IP you want when resolving,
which is convenient when IPv6 is not available on your network. Default
option is "ipv6".
I'm also confused because in the title you said "Lua only". Or maybe you in fact wanted the doc to outline the fact that httpclient is also used for http requests from Lua ? Or maybe update the lua-api doc to indicate that the httpclient is subject to settings in the global section starting with "httpclient" ?
@NickMRamirez These options configure the httpclient, it has nothing to do with lua. One can use the httpclient into lua scripts, but that's not the only usecase. I can add a line which states that it configures the "httpclient" for each option though.
Here's the original info I got about the customer.
A customer had an outage because they believe the default httpclient.resolvers.id applied to server lines and the dns was not resolving properly.
I can try to get more details?
My interpretation is that the customer thought that by configuring httpclient.resolvers.id they could affect DNS resolution in a backend section, but that is not the case?
That's interesting about Lua. That's my misunderstanding then. I thought that these config settings applied only to Lua.
I think I can help understanding the misunderstanding :)
The configuration manual only explains how to configure DNS and SSL for httpclient, but not what haproxy features actually use httpclient.
I'm not a stranger to haproxy but I did not know anything about the httpclient (I was confused a few times by it, and then put it into "fancy LUA stuff/ignore" category in my mind).
I knew that it doesn't relate to actual frontend -> backend HTTP traffic, otherwise it would have come up more often for me. I also knew that DNS for backend server's is configured very differently, so I was aware that it would NOT be related to data plane / http forwarding.
But HTTP is core functionality in haproxy and the term httpclient is completely generic, a new user could easily be confused by it without historic knowledge and then get sidetracked.
I did finally figured it out, by reverse searching git commits I found that one of the first commits regarding httpclient talks about a CLI feature, which finally did point my to the management doc (which I always forget to checkout when I'm looking for something that I don't already know is related to management).
So it is primarily a CLI feature and it can also be used from LUA. That is the information that we need to feed back into the configuration docs, so users understand that it is unrelated to core http forwarding but a (very) auxiliary feature, like explained in the management docs:
http://docs.haproxy.org/2.8/management.html#9.3-httpclient
Launch an HTTP client request and print the response on the CLI. [...] It's only meant for debugging.
I can come up with a RFC patch next week. As you know, doc patches are always best when they come from people that did not understand the thing in the first place.
Ah many thanks for the explanation Lukas.
That's very interesting, because the httpclient is actually not really a CLI feature, it should rather be seen as an internal library for anything that needs to perform HTTP requests outside. The first expected user was the ACME client for LetsEncrypt, but after various shortcomings were identified that approach for now was delayed, then the Lua code because accessing external HTTP stuff from Lua was a pain, everything had to be done entirely by hand with no support for chunked responses etc. We wanted to rely on this for service discovery instead of DNS, though my current opinion on the subject evolved to moving anything discovery-related outside of the process to maintain multiple LBs in sync (but we'll see how all that evolves). Next steps will probably be gRPC when some use cases come for that, then a native ACME client running in the master or another auxiliary process, or logging in JSON over HTTP, etc. We've thought that maybe some advanced HTTP health checks might make use of it, though it looks a bit overkill for something as simple as health checks.
In fact the implementation on the CLI was solely meant as a debugging feature aimed at making sure the generic code works fine, because it was not realistic to imagine implementing the client code and replacing other implementations all at once, and we still needed a trivial service to make use of it.
I easily understand how the doc is unclear about that. I indeed think that some introductory paragraphs should explain that it's not on the datapath and that it should be seen as a generic client used by various utility functions in haproxy (and enumerate them).
I sent an RFC DOC PATCH on the ML, creating a new doc section for httpclient and adding a introductory paragraph :
https://www.mail-archive.com/[email protected]/msg44545.html