Customize http.Client and/or http.Transport

[AnalogJ]

It would be nice to be able to customize the http.Client and/or http.Transport.

I'm using containers/image to build a tool that allows us to sync images servers behind a bastion host. One of the things we need is the ability to set Transport: &http.Transport{Proxy: http.ProxyURL(proxyUrl)} in the http.Client. We'd like to do this dynamically, rather than use the HTTP_PROXY env variable, as the source and destination may use different proxy urls. It would also be nice if we could completely configure the http.Client so we could use a tool like https://github.com/seborama/govcr or ttps://github.com/dnaeon/go-vcr for recording & playback of HTTP requests/responses during testing.

[mtrmac]

Thanks for your report.

For now, just linking earlier #537 .

[AnalogJ]

Thanks for the link @mtrmac

The solution proposed in that PR is basically what I'm looking for, the ability to customize the HTTP Client. From what I'm reading in your comments, there's some concerns about security and configuration overriding the sane defaults.

That’s another point in favor of a more localized HTTPProxy option. Or, maybe, instead of having a straight override which just replaces the http.Client, there could be an “edit hook” (vaguely similar to httpWrapper in #254), which would still leave all of the options supported and used by default, but allow callers to make specific localized edits to the http.Client without affecting the other options — but callers still could replace the implementation wholesale if they really wanted to.

I'm curious if you ever did implement the edit hook option?

[frittentheke]

I arrived at this issue wondering if it was possible to set a proxy server per registry?

While the global http_proxy, https_proxy, and no_proxy variables should definitely be the default (as @jamescassell said in https://github.com/containers/image/pull/537#issuecomment-482712800), there can be cases in which only one registry needs this proxy.

Already having configuration files with containers-registries.conf / containers-registries.conf.d, it's seems quite obvious to also be able to configure settings like proxies or individual TLS CA certs there. Just like with insecure.

[mtrmac]

I’m generally thinking that the proxy requirements are global per machine, and the same for all software running on that machine; would they ever be specific to container registries?

Technically, having a proxy specified in registries.conf could work, sure.

[frittentheke]

I’m generally thinking that the proxy requirements are global per machine, and the same for all software running on that machine; would they ever be specific to container registries?

That might be the case in some cases, and those are well covered with the env vars. But how does one cleanly realize policies to select targets to use the proxy for? So lets say only a certain registry requires a proxy to be reached? I imagine a SOCKS proxy provided by an SSH tunnel to reach a private registry. Browsers have the ancient proxy.pac, but that surely is not a nice alternative, even if that was usable for the Golang HTTP client. I'd rather just configure a proxy for that registry, like I'd set a CA certificate or any other target specific setting.

In short I doubt the reasons for a proxy / relay being required ro reach a registry are 100% aligned with those for other e.g. company proxies.