reqwest icon indicating copy to clipboard operation
reqwest copied to clipboard

No way to set resolver or pre-resolved IP address per request.

Open AMDmi3 opened this issue 7 months ago • 2 comments

My use case is a generic link checker for repology.org. For that I need precise control over HTTP client for predictable and stable behavior and performance. Among other things, I handle domain resolution manually, and feed pre-resolved address to HTTP client for each request.

The problem is that it's not possible to specify an address (or equally a resolver which resolves to a predefined address) during request construction, so I have to build a Client for each request. I've noticed that with native-tls this takes excessive amount of CPU spent in native_tls::TlsConnectorBuilder::build:

flamegraph-native-tls

I've tried switching to rustls backend and it's much better, but I suspect there's its own overhead.

I've glanced through reqwest code and I see that a lot of stuff takes resolver in Client construction, so I believe it won't be trivial to move this setting to Request construction phase. Alternatively, it may be that we can leave with per-request client, if native-tls overhead is fixed, like maybe caching of whetever openssl is loading during client init.

AMDmi3 avatar May 06 '25 16:05 AMDmi3

Related #2641

seanmonstar avatar May 06 '25 18:05 seanmonstar

Hi @seanmonstar, I took a look at this. It appears that DynResolver is part of our HttpConnector, and making it per-request would require significant changes. Would it be a good idea to build the HTTP connector each time? Do you have any suggestions?

Xuanwo avatar May 07 '25 06:05 Xuanwo