running out of requesting address under performance testing

[runvyang]

Setup

os: macOS Catalina Version 10.15.7 go version go1.20.1 darwin/amd64 commit: 1bd05fa3de1c2526a6944e9018be6d7753973314 deploy: examples/hello weaver multi deploy weaver.toml benchmark command: ab -n 100000 -c 8 -k http://10.79.161.90:12345/hello?name=world

Issue:

around finishing 40000 requests, it would throw errors like "http: proxy error: dial tcp 127.0.0.1:54178: connect: can't assign requested address"

Investigation

• found the dial is from http proxy that launched by weaver, and default transport does not limit MaxConnsPerHost, connections were massively created when under pressure;

• tried to set transport args MaxConnsPerHost and MaxIdleConnsPerHost to a high number, tried (100, 100), (150, 100), (100, 150), no longer seeing the same error above, but a new issue come out "panic: net/http: internal error: connCount underflow", this one seems to come up intermittently, see https://github.com/golang/go/issues/38172

• tried the same on a Debian 5.4.143, however the issue didn't reproduce, it seemed to be macOS related.

Questions

• what do you suggest to fix it?

[runvyang]

Okay, it seems i can no longer reproduce "panic: net/http: internal error: connCount underflow", probably it was under critical connection resource saturation.

for those who encounters erorr "connect: can't assign requested address", set MaxConnsPerHost and MaxIdleConnsPerHost to values like (150, 100) the DefaultTransport in file internal/proxy/proxy.go should help.

[rgrandl]

@runvyang can you take another look at #144 ?

[liufuyang]

I encountered the same problem and I seem need to set MaxIdleConnsPerHost as 100 to get over the problem.

I left MaxIdleConns as 100 as well as it is the default transport made for httputil.ReverseProxy is 100 see here and here.

@runvyang do you have time to update your PR? Otherwise, I can also come up with a similar fix.