net/http: configurable error message for Client sent an HTTP request to an HTTPS server.

[mzky]

What version of Go are you using (go version)?

$ go version
go version go1.17.2 linux/amd64

Does this issue reproduce with the latest release?

yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/root/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn"
GOROOT="/var/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/var/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.2"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/root/go/src/net/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1909543103=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Start the HTTPS service written in golang, and the user accesses the HTTP address.

What did you expect to see?

Display localized language information or customize html page or automatically jump to https address.

What did you see instead?

1. Chrome and Firefox: display the following information

2. Other browsers: The IE browser prompts a 400 error, and the instructional content is not displayed. Browsers using the Chromium kernel do not display any visible content. Users will not be able to understand the current situation.

[thanm]

Thanks for raising the issue.

From my read this sounds more like a feature request as opposed to a bug, correct?

@neild

[neild]

The feature request is to respond to an HTTP request on an HTTPS port with a configurable error message.

Testing several major websites (www.google.com, www.amazon.com, www.microsoft.com), none of them respond with an error message to an HTTP request on port 443. Two close the connection without response, one leaves the connection open but does not respond.

I have not checked any other HTTPS server implementations (Apache, nginx, etc.) to see how they handle this condition. It would be interesting to know if any attempt to report an error to the peer in this case.

In the absence of evidence that this is a common feature, I don't think we should add this.

[mzky]

Hi ! This is a feature request, not a bug.

I found some examples of Nginx configuring HTTP redirection to HTTPS：https://linuxize.com/post/redirect-http-to-https-in-nginx/

1. Old version Parameters： ··· rewrite ^(.*)$ https://$host$1 permanent; ···
2. New version Parameters： ··· return 301 https://$server_name$request_uri;
   ···

I want to replace nginx's auto-jump feature with Golang. Of course, this is just a suggestion, the changes will not affect the original functions.

[seankhliao]

HTTP redirection is different from the filed issue. It's already possible by listening on both HTTP and HTTPS ports and handling the redirection within the handlers

go http.ListenAndServe(":80", http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
        // handle redirect here
}))
http.ListenAndServeTLS(":443", "...", "...", nil)

[mzky]

HTTP redirection is different from the filed issue. It's already possible by listening on both HTTP and HTTPS ports and handling the redirection within the handlers

go http.ListenAndServe(":80", http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
        // handle redirect here
}))
http.ListenAndServeTLS(":443", "...", "...", nil)

Yes, different port forwarding is also possible, but I need a forwarding on the same port

[mzky]

They all jump to https with the same port:

[mzky]

Go language simulates google：

[neild]

They all jump to https with the same port:

An https:// URL with no port uses a default port of 443, not 80. All these are redirecting from HTTP on port 80 to HTTPS on port 443.

[hktalent]

I would also like to make some comments on this issue

• First of all, since the server can output "Client send an HTTP request to an HTTPS server", it should give the control of the output to the developer instead of burying him deeply

• Secondly, to solve this problem, we should not use nginx or listen to port 80 more, which are very cumbersome

[hktalent]

Yes, any core modifications don't seem to be standard-compliant, but, that's the truth, we need a way to intercept when "Client send an HTTP request to an HTTPS server" happens, or have a kind of friendly wrapper that allows him to Controllable output, because "Client send an HTTP request to an HTTPS server" directly output to the browser is very unfriendly, and it is uncontrollable and unexpected

[mitar]

I would also like that this is configurable. Maybe http.Server could get the following field:

TLSHandshakeBadRequest func(conn net.Conn)

And if not defined, current:

io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")

is called. Otherwise TLSHandshakeBadRequest(re.Conn) is called.

Open to other field names.

[martin0104]

i have same problem that response content not friendly, i wish can configurable the content

[bddjr]

The nginx example you provided is incorrect, it should be like this:

# port 443
error_page 497 https://$host$request_uri;

# other ports
error_page 497 https://$host:$server_port$request_uri;

Then, attempting to access the HTTPS port using HTTP will return a 302 redirect.
For example:
http://q8p.cc:443

---

For this, I wrote a script to implement browser redirection without affecting the HTTP 400 status code.
https://github.com/bddjr/go-https-port-auto-redirect-for-http

HTTP/1.1 400 Bad Request
Content-Type: text/html
Connection: close

<!-- Client sent an HTTP request to an HTTPS server. -->
<script> location.protocol = 'https:' </script>

[bddjr]

Maybe we can read the "Host" request header, then return 307 status code

The following is a number of debugging screenshots, not yet implemented the above functions

src/crypto/tls/conn.go

src/net/http/server.go

test.go

[bddjr]

I did it!
Next, I will submit a pull request.

[gopherbot]

Change https://go.dev/cl/564997 mentions this issue: net/http: configurable error message for Client sent an HTTP request …

[neild]

It looks like nginx responds to an HTTP request on an HTTPS port with an error 497 (an nginx-specific code), and permits code-specific error handler overrides. The example nginx configuration from above is interesting, because it seems to indicate that nginx is parsing the full HTTP request in this case:

error_page 497 https://$host$request_uri;

This points at a question: Is the desired feature request to customize the static message we send when responding to a misdirected HTTP request, or to respond with a redirect to the correct location? The latter is substantially more complicated to implement, because it requires effectively restarting the connection.

If were were to add a feature, I could imagine something like:

type Server struct {
  // HTTPOnHTTPSPortErrorResponseString is sent on an HTTPS connection
  // which receives what looks like an HTTP request.
  HTTPOnHTTPSPortErrorResponse string
}

But that's static, and it sounds like a common use case is a redirect. So maybe what people want is:

type Server struct {
  // HTTPOnHTTPSPortErrorHandler handles HTTP requests sent to an HTTPS port.
  HTTPOnHTTPSPortErrorHandler Handler
}

Although in that case, what happens when someone inevitably sets Server.Handler and Server.HTTPOnHTTPSPortErrorHandler to the same thing? Does that work? Should that work? Are there edge cases where it fails? I think implementation for this would also be complicated; unless I'm wrong, I don't think this is something we'd want to do.

A middle ground, suggested by @mitar above, is essentially a roll-your-own handler where we hand the net.Conn to the user:

type Server struct {
  TLSHandshakeBadRequest func(conn net.Conn)
}

You're going to want the already-consumed bytes from the net.Conn as well though, no?

I don't have any clear answers here. There's been a fair bit of discussion on this issue over time, but half of it is about cases other than an HTTP request sent to an HTTPS port. (For example, redirecting from HTTP on port 80 to HTTPS on port 443, which of course is something you can do today.) Some of the example use cases (such as that nginx config) involve more than just customizing the error string.

Would a statically customizable response string be enough? That's at least implementable without adding a ton of complexity.

[bddjr]

A static string can be redirected in the browser using JavaScript,
but it doesn't fit the 301 status code redirect that some people want

"HTTP/1.1 400 Bad Request\r\nContent-Type: text/html\r\nConnection: close\r\n\r\n<!-- Client sent an HTTP request to an HTTPS server. -->\n<script> location.protocol = 'https:' </script>\n"

HTTP/1.1 400 Bad Request
Content-Type: text/html
Connection: close

<!-- Client sent an HTTP request to an HTTPS server. -->
<script> location.protocol = 'https:' </script>

[mitar]

@neild Yea, I realized this in https://go-review.googlesource.com/c/go/+/564997 as well. I think we could have:

type Server struct {
  TLSHandshakeBadRequest func(conn net.Conn, recordedBytes []byte)
}

But I think those recorded bytes are really optional and it might be hard for later on for implementation to change its behavior. If we look at current code, it does not use those bytes, it just writes to the connection:

io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")

So I was envisioning that the TLSHandshakeBadRequest would do the same, only maybe write a Location header redirect.

I am not sure how much utility is in inspecting recordedBytes?

[bddjr]

I think it's possible to provide multiple interfaces for different use cases, like TLSConfig's NameToCertificate and GetCertificate

(I think the relevant configuration should be written in TLSConfig)

type Config struct {
  // HttpOnHttpsPortErrorResponseString is sent on an HTTPS connection
  // which receives what looks like an HTTP request.
  HttpOnHttpsPortErrorResponse string

  // HttpOnHttpsPortErrorRedirect is sent on an HTTPS connection
  // which receives what looks like an HTTP request.
  // If true, 307 redirect will be sent
  HttpOnHttpsPortErrorRedirect bool

  // HttpOnHttpsPortErrorHandler handles HTTP requests sent to an HTTPS port.
  HttpOnHttpsPortErrorHandler func(conn net.Conn, requestBytes []byte)

Priority is from bottom to top

[bddjr]

I am not sure how much utility is in inspecting recordedBytes?

307 redirect

var compiledRegExp_httpHost = regexp.MustCompile(`\r\nHost: \S+`) // "\r\nHost: local.q8p.cc:5678"
var compiledRegExp_httpPath = regexp.MustCompile(`/\S*`)          // "/index.html"

func httpOnHttpsPortErrorHandler (conn net.Conn, requestBytes []byte) {
    requestString := string(requestBytes)
    Host := compiledRegExp_httpHost.FindString(requestString) // "\r\nHost: local.q8p.cc:5678"
    if Host == "" {
        io.WriteString(conn, "HTTP/1.1 400 Bad Request\r\n"+
            "\r\n"+
            "Missing required Host header.\n",
        )
        return
    }
    Host = Host[8:]                                             // "local.q8p.cc:5678"
    Path := compiledRegExp_httpPath.FindString(requestString) // "/index.html"
    io.WriteString(conn, "HTTP/1.1 307 Temporary Redirect\r\n"+
        "Location: https://"+Host+Path+"\r\n"+
        "Connection: close\r\n"+
        "\r\n"+
        "Client sent an HTTP request to an HTTPS server.\n",
    )
}

HTTP/1.1 307 Temporary Redirect
Location: https://local.q8p.cc:5678/index.html
Connection: close

Client sent an HTTP request to an HTTPS server.

[bddjr]

I just found a more standard request parsing method

req, err := http.ReadRequest(bufio.NewReader(bytes.NewReader(requestBytes)))
if err != nil {
	fmt.Println(err)
	return
}
io.WriteString(conn, fmt.Sprintf(
	"HTTP/1.1 307 Temporary Redirect\r\nLocation: https://%s%s\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n",
	req.Host, req.URL.Path,
))

[mitar]

I see. To allow virtual hosts redirects. And if requestBytes does not contain enough data (not yet all headers), you can still read more from the connection?

[bddjr]

🤔I just tried, it doesn't work, I'll see if there's an interface to continue reading

The request content cannot exceed 576 bytes, otherwise the request cannot be parsed

[bddjr]

I successfully read the remaining bytes from conn

server.go

// net/http: configurable error message for Client sent an HTTP request to an HTTPS server.
// https://go.dev/issue/49310
func (c *conn) httpOnHttpsPortErrorHandler(conn net.Conn, recondBytes []byte) {
	// Read Response string
	badRequestResponse := c.server.TLSConfig.HttpOnHttpsPortErrorResponse
	if badRequestResponse == "" {
		badRequestResponse = "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n"
	}

	// Handler
	if handler := c.server.TLSConfig.HttpOnHttpsPortErrorHandler; handler != nil {
		handler(conn, recondBytes, badRequestResponse)
		return
	}

	// Redirect
	if c.server.TLSConfig.HttpOnHttpsPortErrorRedirect {
		// Read Header
		req, _, err := ReadRequestForHttpOnHttpsPortErrorHandler(conn, recondBytes)
		if err != nil {
			io.WriteString(conn, badRequestResponse)
			return
		}

		// Send Redirect
		io.WriteString(conn, fmt.Sprintf(
			"HTTP/1.1 307 Temporary Redirect\r\nLocation: https://%s%s\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n",
			req.Host, req.URL.Path,
		))
		return
	}

	// Send Response string
	io.WriteString(conn, badRequestResponse)
}

func ReadRequestForHttpOnHttpsPortErrorHandler(conn net.Conn, recondBytes []byte) (req *Request, reqBytes []byte, err error) {
	// Max 4KB
	if len(recondBytes) > 4096 {
		return nil, recondBytes, errors.New("recondBytes too long")
	}

	// Content length may be insufficient, continue reading.
	if len(recondBytes) < 4096 && !bytes.Contains(recondBytes, []byte("\r\n\r\n")) {
		b := make([]byte, 4096-len(recondBytes))
		// Set Timeout 1s
		conn.SetReadDeadline(time.Now().Add(time.Duration(1 * time.Second)))
		n, err := conn.Read(b)
		if err != nil {
			return nil, recondBytes, err
		}
		recondBytes = append(recondBytes, b[:n]...)
	}

	// Read Request
	req, err = ReadRequest(bufio.NewReader(bytes.NewReader(recondBytes)))
	if err != nil {
		return nil, recondBytes, err
	}
	if req.Host == "" {
		return nil, recondBytes, errors.New("missing required Host header")
	}
	return req, recondBytes, nil
}

[bddjr]

If this is enough, I will submit a PullRequest @mitar

TLSConfig

type Config struct {
	// HttpOnHttpsPortErrorResponseString is sent on an HTTPS connection
	// which receives what looks like an HTTP request.
	//   "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n"
	HttpOnHttpsPortErrorResponse string

	// HttpOnHttpsPortErrorRedirect is sent on an HTTPS connection
	// which receives what looks like an HTTP request.
	// If true, 307 redirect will be sent
	HttpOnHttpsPortErrorRedirect bool

	// HttpOnHttpsPortErrorHandler handles HTTP requests sent to an HTTPS port.
	//
	// WriteString use:
	//   io.WriteString(conn, "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")
	// Parse the request header use:
	//   http.ReadRequestForHttpOnHttpsPortErrorHandler(conn, recondBytes)
	HttpOnHttpsPortErrorHandler func(conn net.Conn, recondBytes []byte, badRequestResponse string)

[mitar]

@bddjr Please read contributing guide. I think this proposal has not yet been accepted. Making a pull request is fine to be able to discuss code, but until the proposal is accepted it cannot be merged.

And about your last proposal: personally, I think the Config is overkill, I think we should have only one function callback and not so many options.

Also, what is badRequestResponse in HttpOnHttpsPortErrorHandler?

[bddjr]

Also, what is badRequestResponse in HttpOnHttpsPortErrorHandler?

It will be TLSConfig.HttpOnHttpsPortErrorResponse

	// Read Response string
	badRequestResponse := c.server.TLSConfig.HttpOnHttpsPortErrorResponse
	if badRequestResponse == "" {
		badRequestResponse = "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n"
	}

	// Handler
	if handler := c.server.TLSConfig.HttpOnHttpsPortErrorHandler; handler != nil {
		handler(conn, recondBytes, badRequestResponse)
		return
	}

[bddjr]

✨ I made a go mod hijacking net.Conn to implement this feature

https://github.com/bddjr/hlfhr

var srv *hlfhr.Server

func main() {
	// Use hlfhr.New
	srv = hlfhr.New(&http.Server{
		Addr: ":5678",
		Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			// Write something...
		}),
		ReadHeaderTimeout: 10 * time.Second,
		IdleTimeout:       10 * time.Second,
	})
	// Then just use it like http.Server .

	err := srv.ListenAndServeTLS("localhost.crt", "localhost.key")
	fmt.Println(err)
}

HttpOnHttpsPortErrorHandler

// Default
srv.HttpOnHttpsPortErrorHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	hlfhr.RedirectToHttps(w, r, 302)
})

// Check Host Header
srv.HttpOnHttpsPortErrorHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	switch hlfhr.Hostname(r.Host) {
	case "localhost":
		hlfhr.RedirectToHttps(w, r, 302)
	case "www.localhost", "127.0.0.1":
		hlfhr.Redirect(w, 302, "https://"+hlfhr.ReplaceHostname(r.Host, "localhost")+r.URL.RequestURI())
	default:
		w.WriteHeader(421)
	}
})

// Script Redirect
srv.HttpOnHttpsPortErrorHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "text/html")
	w.WriteHeader(400)
	io.WriteString(w, "<script> location.protocol = 'https:' </script>")
})

// Keep Alive
srv.HttpOnHttpsPortErrorHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Connection", "keep-alive")
	w.WriteHeader(400)
	io.WriteString(w, "Hello hlfhr")
})