Host header rewrite

[danydev]

First of all, let me say that you did an amazing job! Sometimes is really useful to expose my local environment for a quick test, I usually use Charles acting as proxy or ngrok, but expose will be even better since I can reserve my own subdomain!

Now the problem: when accessing my local environment the host header returned by my webserver is hardcoded, so when I expose with expose share mywebsite.local.com:443 and then go to https://something.sharedwithexpose.com I correctly see the web page served by my local environment but I get redirected to https://mywebsite.local.com, this because there is a redirection based in the host header and such header value is https://mywebsite.local.com instead of https://something.sharedwithexpose.com What ngrok offers to overcome this use case is a flag -host-header=rewrite.

Quoting their doc

Rewriting the Host header
When forwarding to a local port, ngrok does not modify the tunneled HTTP requests at all, they are copied to your server byte-for-byte as they are received. Some application servers like WAMP and MAMP and use the Host header for determining which development site to display. For this reason, ngrok can rewrite your requests with a modified Host header. Use the -host-header switch to rewrite incoming HTTP requests.

If rewrite is specified, the Host header will be rewritten to match the hostname portion of the forwarding address. Any other value will cause the Host header to be rewritten to that value.

Rewrite the Host header to 'site.dev'
ngrok http -host-header=rewrite site.dev:80
Rewrite the Host header to 'example.com'
ngrok http -host-header=example.com 80

[aaemnnosttv]

I thought this might be an issue as well, but it seems that this is already happening. When you dump $_SERVER['HTTP_HOST'] you should see the hostname of the URL you shared.

This looks like the same issue as #24 .

[danydev]

Not for me, I see https://mywebsite.local.com in HTTP_HOST

[mpociot]

Hi @danydev

We are actually already doing this. Even though the host name can't be configured manually - maybe that would be helpful. But instead, the hostname gets used when you specify it while sharing:

See https://github.com/beyondcode/expose/blob/e870878bceb84b9f5d8ec6c81bb89078e2489d15/app/Server/Http/Controllers/TunnelMessageController.php#L116

This is the host that you configure when sharing your site.

So expose share foo.test will set the host header to foo.test.

Could you try and see if it makes a difference for you if you do:

expose share https://mysite.test

vs

expose share mysite.test:443

[danydev]

No difference, I still see the 'wrong' header. But now that you're telling me that the tool should already handle it. I suppose it has something to do with my setup. My dev environment is composed by a traefik container acting as reverse proxy to a nginx container.

Looks like ngrok is able to 'override' somehow the redirection that my application-level code is doing to the correct host. I think it's just because they do some additional work to make it working in those cases.

Note how if I go to a path where there are NO redirections the website is correctly served, only when there is a 30x I got redirected to the "wrong” local url.

[mpociot]

Would be interesting to see the headers that ngrok is sending in your specific use case (taken from the local inspector view) compared to what expose is doing.

[danydev]

Looks like my application-level code sees the 'wrong' header with ngrok as well (and that's fine as long as I get redirected to the correct host), so I suppose the 'fix' is done by ngrok while acting as middle-man.

Inspecting the response with a proxy I can tell ngrok response already contains the correct Location/Host headers (Location: https://bbd708a5e84c.ngrok.io/login), so I think it's something they do server-side. Maybe they just replace Host\Location headers manually when such option is on?

Also, not sure if you are asking for something more specific, if so, just tell me what you need to look into.

[mpociot]

Yeah I would be interested in the headers that you see in the local dashboards when you inspect a specific request (the one at http://127.0.0.1:4040 As they contain the headers that get sent to your expose client - without your application level code being in-between (and any proxied servers)

[danydev]

I did not know http://127.0.0.1:4040, so that's the result:

Request:

GET /

Host | myservice.local.it
X-Original-Host | bbd708a5e84c.ngrok.io
X-Forwarded-For | my-ip-address (redacted)

Response:

302 Found
Location | https://myservice.local.it/login

Now, you may say "well, looks like you should be redirected to myservice.local.it". Not really, the curl tells me that:

Location: https://bbd708a5e84c.ngrok.io/login

indeed I get redirected there (to ngrok.io, not to myservice).

So I can't tell you why that debugging tool tells the wrong location, while the actual response inspected with curl contains the right location (and indeed, it works). It may just be that the tool contains what ngrok sees without the actual fix done server side?

[mpociot]

Okay great, could you show me the same header results on expose's dashboard? Just to see if the headers are the same

[danydev]

Yes, they look the same

GET /

Host myservice.local.it:443
x-original-host xudu45wyce.sharedwithexpose.com
x-forwarded-for my-ip-address

Response

302 Found
Location https://myservice.local.it/login

but the actual curl with expose says

Location: https://myservice.local.it/login

[Sebastiangperez]

I have the same issue , when i perform a login , redirects to my localdomain with the page expired message.

[stuba]

Same here. I'm using Pomerium as proxy. I did test Ngork and it did seem to work better in this case, maybe it is working more like a "real proxy" service.

Now I did manage to overcome this problem, just for a sake of testing, with few changes.

• Add xyz.sharedwithexpose.com to your /etc/hosts pointing 127.0.0.1
• Add separate route to my Pomerium config from URL xyz.sharedwithexpose.com
• Add to my Nginx config 'fastcgi_param SERVER_NAME' xyz.sharedwithexpose.com
• My Pomerium requires valid cert so I needed to open xyz.sharedwithexpose.com once and accept failing cert

So most likely all you that have similar problems have a look at your proxies and test connections with curl that Expose dashboard provides to you. That helped me to understand this issue a bit. I'm not a real proxy wizard and there might much easier solutions for this, like Pomerium preserve_host_header. Of course if Expose could work a bit better on this matter, but I'm not that sure how easily it could be achieved. Maybe there is more clever guys who could provide some good ideas..?

Nice tool overall guys!

[stuba]

Just as an update for my previous post. Now it all works perfectly. Key was to make sure original URL is passed correctly to application server. So I would not call this a bug, but just thing that might be worth document a bit better.

Resolution will be up to you setup, how it worked out for me. First I had to make sure Pomerium route has preserve_host_header = true. Depending on source my app will receive requested host from header as host or X-Original-Host. Then it was just a matter of selecting correct host and pass it over to fastcgi.

map $http_x_original_host $my_real_host {
    default       $http_x_original_host;
    ''            $http_host;
}
.
.
.
fastcgi_param SERVER_NAME $my_real_host;
fastcgi_param HTTP_HOST   $my_real_host;

So now my application will receive host used for original request and it will generate correct URLs depending where requests are coming from and I do not have to modify my application source code at all. Hope this helps somebody to with similar problems.

[mpociot]

I just added the X-Forwarded-Host to expose, so hopefully servers can just make use of this. Because host header rewrites "just work" based on the site that you share. Maybe it might be an interesting option to manually specify the host header when sharing a site, but since you share URLs, you can just share the URL that contains the correct hostname.

[danydev]

Just tested, and it did not solve my problem.

Just to clarify: I could easily fiddle with traefik (or nginx) and headers to fool my web server and make it works, but still would require me make operational changes for the tool. Something that is not super bad, but still ngrok just works out of the box.

Looks like ngrok 'does the magic' doing some headers overwrite as soon as it 'proxies' the response. I just debugged my application-level code and it returns a 302 on https://myservice.local.it/login so ngrok does its trick just later in the 'client', given that the web server has the 'wrong' host the whole time, but still the actual response contains the right url.

[tomb1n0]

@danydev did you ever figure out a workaround for this? I love the look of Expose but also using Traefik

[danydev]

The key is to replace Host with X-Original-Host in your web server. Traefik does not allow dynamic header rewrite (I think it's in the works), but you can still hardcode it.

[SleeplessByte]

If I recall correctly, in order for systems such as Laravel, Rails, or reverse proxies etc. to pick this up correctly, it's not only X-Forwarded-Host that should be added, but also X-Forwarded-For and X-Forwarded-Proto. Might as well also add the Forwarded header.

[guilhermeagirardi]

Just as an update for my previous post. Now it all works perfectly. Key was to make sure original URL is passed correctly to application server. So I would not call this a bug, but just thing that might be worth document a bit better.

Resolution will be up to you setup, how it worked out for me. First I had to make sure Pomerium route has preserve_host_header = true. Depending on source my app will receive requested host from header as host or X-Original-Host. Then it was just a matter of selecting correct host and pass it over to fastcgi.

map $http_x_original_host $my_real_host {
    default       $http_x_original_host;
    ''            $http_host;
}
.
.
.
fastcgi_param SERVER_NAME $my_real_host;
fastcgi_param HTTP_HOST   $my_real_host;

So now my application will receive host used for original request and it will generate correct URLs depending where requests are coming from and I do not have to modify my application source code at all. Hope this helps somebody to with similar problems.

Very goooood! Thank you! 😃

[danydev]

I'm not sure it's good to close the issue here. While we can actually fiddle with webserver\proxy parameters to get it right, ngrok doesn't need to do it in order to give the same functionality.

[mpociot]

@danydev I thought that this issue was fixed by now. Are you still experiencing this issue?

If you do, I would highly appreciate an easy-to-reproduce repository so that I can see how this can be fixed inside of Expose.

[danydev]

Yes, it's still happening in 1.5.1. I'll try to create a mini-application in the next week or two, so you can actually reproduce it.

[sschlein]

Closing this issue because it's old 🙈