Using remap to whitelist hostnames is not working with HTTPS on some hosts

[malhomaid]

I'm using trafficserver 9.1.1 and running it as forward proxy, when I set remap_required to 1, and added this line map https://example.com https://example.com @action=allow in remap.config.

My question is this the *correct way to do whitelisting be cause I don't want to do any mapping I just want to whitelist? also this works fine but we are currently facing the issue below with 20% of our domains.

When I try to connect with remap_required set to 0 it works fine

curl -x localhost:8080 https://example.com -v
*   Trying localhost:8080...
* Connected to (nil) (localhost) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to example.com:443
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> User-Agent: curl/7.81.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< Date: Wed, 05 Jul 2023 12:15:35 GMT
< Proxy-Connection: keep-alive
< Server: ATS/9.1.1
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=example.com
*  start date: Jun 29 23:34:04 2023 GMT
*  expire date: Sep 27 23:34:03 2023 GMT
*  subjectAltName: host "example.com" matched cert's "example.com"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET / HTTP/1.1
> Host: example.com
> User-Agent: curl/7.81.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Wed, 05 Jul 2023 12:15:36 GMT
< Server: Apache
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
< 
* Connection #0 to host (nil) left intact

When I try to connect with remap_required set to 1 and add map https://example.com https://example.com @action=allow it get stuck at TLS Client hello and then I receive a timeout.

curl -x localhost:8080 https://example.com -v
*   Trying localhost:8080...
* Connected to (nil) (localhost) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to example.com:443
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> User-Agent: curl/7.81.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< Date: Tue, 04 Jul 2023 15:12:51 GMT
< Proxy-Connection: keep-alive
< Server: ATS/9.1.1
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.0 (OUT), TLS header, Unknown (21):
* TLSv1.3 (OUT), TLS alert, decode error (562):
* error:0A000126:SSL routines::unexpected eof while reading
* Closing connection 0
curl: (35) error:0A000126:SSL routines::unexpected eof while reading

[mlibbey]

FWIW, we have proxy.config.url_remap.remap_required: 1 but, our mapping rules have the origin's port -- like map https://example.com https://example.com:443 without the port, the requests fail.

[malhomaid]

@mlibbey It worked I added it to both client and origin like this map https://example.com:443 https://example.com:443 Thank you so much! @mlibbey I don't know why it was working with more than 20 domain(without adding 443 port) 😅

[mlibbey]

Interesting -- we only have the port on the origin side of the mapping.

Also, FYI, took me forever to get CONNECT over https working -- the curl incarnation was curl -sD- -o/dev/null -x https://{ATSNameThatHasACert} https://www.example.com/

[malhomaid]

@mlibbey with me curl and other clients are working without any special config

[malhomaid]

Docs seems to be clear but I assumed that I don't need to define 443 port because it's a default for the protocol, so should this be added to the docs? or I should close the issue

[randall]

@mlibbey I'm not sure the port number is absolutely required. I believe I added that to work around our configuration systems constraints (cannot have a inbound match the outbound literally).

[mlibbey]

I did test that in our environment -- not having the port on the origin side of the mapping leads to a 403 Tunnel Forbidden on the "inner" connection"; while having it leads to 200s on the inner side. There could be other settings that are causing that though.

[mlibbey]

mhomaid1 -- could you test with and without port on the inbound side of the mapping? The community always welcomes pull requests especially for documentation improvements!

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. Marking it stale to flag it for further consideration by the community.