🔥 Feature: Add custom http proxy for http client in proxy middleware

[jj0e]

Description

The proxy middleware is a great way to make your server act as a proxy in order to make changes to requests, etc. However, it lacks the ability to control the dialer of http client directly specifically for using http proxies. What do I mean by this?

Right now, this diagram can be used to describe the middleware:

Proxy in this case is the fiber server using the proxy middleware. Lets say the IP address of Proxy is at example.com:1337, when Bob receives the proxy request via Alice, the IP address that he will see is example.com:1337, not the IP address of Alice. This presents an issue for those who are handling a high load of requests and proxying them to other sites which may contain rate limiting or web application firewalls resulting in 429 Too Many Requests errors.

This error is not at the fault of Alice but rather Proxy. It may seem to be an odd or rare solution, however in the market that I'm operating, it is common practice to implement another proxy layer. The logic is similar to the already implemented load balancer, however there are certain cases where even the load balancer is not enough to handle the load. In this case, we require the user to send a custom HTTP proxy in their request (header, cookie, or payload). We then take this proxy and use it inside of the proxy middleware to spoof the IP address of the server. This can be demonstrated like so:

This pull requests allows us to create another proxy layer in between Proxy and Bob. There are no added dependencies. It is possible to implement a custom HTTP proxy via Fasthttp's fasthttpproxy package which is already implemented inside of Fasthttp. It edits the clients Dial which in the case of this middleware is not being used for anything else.

Usage

Without authentication

proxy.WithHttpProxy("example.com:1337")

// Will use example.com:1337 as the request IP instead of the server IP
if err := proxy.Do(c, url); err != nil {
    log.Fatal(err)
}

With user/pass authentication

proxy.WithHttpProxy("user:[email protected]:1337")

// Will use user:[email protected]:1337 as the request IP instead of the server IP
if err := proxy.Do(c, url); err != nil {
    log.Fatal(err)
}

[welcome[bot]]

Thanks for opening this pull request! 🎉 Please check out our contributing guidelines. If you need help or want to chat with us, join us on Discord https://gofiber.io/discord

[ReneWerner87]

@jj0e could you check my last comment

[jj0e]

@jj0e could you check my last comment

I'm having trouble creating a reverse proxy to use in the test. I don't have much time to work on this although I did want to get the ball rolling. I will get to it ASAP

[ReneWerner87]

Please check the existing tests https://github.com/gofiber/fiber/blob/master/middleware/proxy/proxy_test.go#L31

[ReneWerner87]

@jj0e Please check the existing tests https://github.com/gofiber/fiber/blob/master/middleware/proxy/proxy_test.go#L31

and adapt them for the new tests

[ReneWerner87]

@jj0e please check my review comments

[ReneWerner87]

@jj0e any progress here ?

[jj0e]

No, I have looked into implementing tests and I'm stuck. I do not know how to correctly implement.

[efectn]

No, I have looked into implementing tests and I'm stuck. I do not know how to correctly implement.

It looks like we need to create simple tunnel proxy to create unit tests. I'll try to do it weekend.

[efectn]

Overridden by https://github.com/gofiber/fiber/pull/2117