httpx icon indicating copy to clipboard operation
httpx copied to clipboard

Add `httpx.NetworkOptions` configuration.

Open lovelydinosaur opened this issue 1 year ago • 4 comments
trafficstars

Refs https://github.com/encode/httpx/issues/947#issuecomment-1881173378

Here's how the proposed documentation looks...


Network Options

There are several advanced network options that are made available through the httpx.NetworkOptions configuration class.

# Configure an HTTPTransport with some specific network options.
network_options = httpx.NetworkOptions(
    connection_retries=1,
    local_address="0.0.0.0",
)
transport = httpx.HTTPTransport(network_options=network_options)

# Instantiate a client with the configured transport.
client = httpx.Client(transport=transport)

Configuration

The options available on this class are...

connection_retries

Configure a number of retries that may be attempted when initially establishing a TCP connection. Defaults to 0.

local_address

Configure the local address that the socket should be bound too. The most common usage is for enforcing binding to either IPv4 local_address="0.0.0.0" or IPv6 local_address="::".

socket_options

Configure the list of socket options to be applied to the underlying sockets used for network connections. For example, you can use it to explicitly specify which network interface should be used for the connection in this manner:

import httpx

socket_options = [(socket.SOL_SOCKET, socket.SO_BINDTODEVICE, b"ETH999")]

network_options = httpx.NetworkOptions(
    socket_options=socket_options
)

uds

Connect to a Unix Domain Socket, rather than over the network. Should be a string providing the path to the UDS.


Design options that we have here...

  • We could allow a httpx.Client(network_options=...) shortcut, in the same way we do for eg. ssl_context=..., proxy=..., limits=..., version=....
  • We could have Limits and NetworkOptions all be part of the same single config class. Including max_keepalive_connections, max_connections, keepalive_expiry on this class.

lovelydinosaur avatar Jan 12 '24 12:01 lovelydinosaur

I would like to ask that something be added to the documentation at advanced usage, please. I would like you to add something such as: "This is how you force httpx to use IPv4 even if it can use IPv6. Using IPv6 is the default. My proposed change will make it so that if somebody searches for "IPv4", it will be found. Currently, it was very hard to find until I was told what to look for. Of course, once I was told what to look for, it was immediately obvious. Thank you.

Commendation to karpetrosyan for his quick response to my question!

My earlier comment about IPv4 mapped IPv6 addresses is not a function of httpx. It is a "feature" of nginx. I was able to verify using tcpdump that httpx works correctly as explained to me.

jeffsilverm avatar Jan 16 '24 10:01 jeffsilverm

Here is the socket_options example that we might want to use here.

karpetrosyan avatar May 17 '24 17:05 karpetrosyan

I have added an example for the socket_options parameter. I believe we are now ready to merge this PR.

karpetrosyan avatar Jun 13 '24 15:06 karpetrosyan

Maybe we should also add a changelog

karpetrosyan avatar Jun 13 '24 15:06 karpetrosyan