Support NTLM/Kerberos upstream proxy authentication

[SamuelePilleri]

I'm trying to create a local proxy service based on the WindowsService example to overcome limitations in my company.

Every PC is set up with a PAC URL that contains what proxy to use for each destination. This is what a correct request to the outside world looks like:

$ curl --proxy {upstreamproxy}:8080 --proxy-ntlm --proxy-user ":" --verbose https://ipinfo.io
* Trying {omitted}:8080...
* Connected to {upstreamproxy} ({omitted}) port 8080 (#0)
* allocate connect buffer
> CONNECT ipinfo.io:443 HTTP/1.1
> Host: ipinfo.io:443
> Proxy-Authorization: NTLM {omitted}
> User-Agent: curl/7.83.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 407 authenticationrequired
< Date: Wed, 04 Jan 2023 10:36:50 GMT
< Content-Type: text/html
< Cache-Control: no-cache
< Content-Length: 0
< X-Frame-Options: deny
< Proxy-Connection: Keep-Alive
< Proxy-Authenticate: NTLM {omitted}
<
> CONNECT ipinfo.io:443 HTTP/1.1
> Host: ipinfo.io:443
> Proxy-Authorization: NTLM {omitted}
> User-Agent: curl/7.83.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 200 Connection established
<
> GET / HTTP/1.1
> Host: ipinfo.io
> User-Agent: curl/7.83.1
> Accept: */*
>
< HTTP/1.1 200 OK
< {ipinfo.io response headers, omitted}

Based on the example provided, this is what I've written:

protected override void OnStart(string[] args)
{
	proxyServerInstance = new(userTrustRootCertificate: false) {
		CheckCertificateRevocation = X509RevocationMode.NoCheck,
		ConnectionTimeOutSeconds = 30,
		Enable100ContinueBehaviour = true,
		EnableConnectionPool = true,
		EnableTcpServerConnectionPrefetch = true,
		EnableWinAuth = true,
		ForwardToUpstreamGateway = true,
		MaxCachedConnections = 2,
		ReuseSocket = true,
		TcpTimeWaitSeconds = 30,
		EnableHttp2 = true,
		NoDelay = true,
		ThreadPoolWorkerThread = Environment.ProcessorCount,
		ExceptionFunc = ProxyException,
	};
	
	proxyServerInstance.CertificateManager.SaveFakeCertificates = false;

	var explicitEndPointV4 = new ExplicitProxyEndPoint(IPAddress.Loopback, ListeningPort, false);
	proxyServerInstance.AddEndPoint(explicitEndPointV4);

	var explicitEndPointV6 = new ExplicitProxyEndPoint(IPAddress.IPv6Loopback, ListeningPort, false);
	proxyServerInstance.AddEndPoint(explicitEndPointV6);

	proxyServerInstance.Start();

	#if DEBUG
	Console.WriteLine($"Service Listening on port {ListeningPort}");
	#else
	ProxyServiceEventLog.WriteEntry($"Service Listening on port {ListeningPort}", EventLogEntryType.Information);
	#endif
}

However the exception Upstream proxy failed to create a secure tunnel is raised. https://github.com/justcoding121/titanium-web-proxy/blob/902504a324425e4e49fc5ba604c2b7fa172e68ce/src/Titanium.Web.Proxy/Network/TcpConnection/TcpConnectionFactory.cs#L519

Looking at the code it may depend on how Windows authentication is handled. https://github.com/justcoding121/titanium-web-proxy/blob/902504a324425e4e49fc5ba604c2b7fa172e68ce/src/Titanium.Web.Proxy/ResponseHandler.cs#L40-L47

I guess the problem here is the upstream proxy replying with 407 rather then 401, but I have limited debugging capabilities (work PC) and it's hard to investigate further.

Can someone please help me troubleshoot this? I'm using .NET Core 6 (can't install full Visual Studio) and Titanium 3.2.0.

[SamuelePilleri]

Upon further inspection it seems to me that the library does not handle the case when the upstream proxy server supports NTLM/Kerberos authentication.

I'm willing to implement it, I just need some guidance on what may already be in place to support it in this huge code base.

On the protocol side, it should no be very difficult:

$ curl --proxy {upstreamproxy}:8080 --verbose https://ipinfo.io
* Trying {omitted}:8080...
* Connected to {upstreamproxy} ({omitted}) port 8080 (#0)
* allocate connect buffer
* Establish HTTP proxy tunnel to ipinfo.io:443
> CONNECT ipinfo.io:443 HTTP/1.1
> Host: ipinfo.io:443
> User-Agent: curl/7.83.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 407 authenticationrequired
< Date: Thu, 05 Jan 2023 17:59:47 GMT
< Content-Type: text/html
< Cache-Control: no-cache
< Content-Length: 3741
< X-Frame-Options: deny
< Proxy-Connection: Keep-Alive
< Proxy-Authenticate: NTLM
< Proxy-Authenticate: Basic realm="{omitted}"
<
* Closing connection 0

As you can see the proxy server itself replies with supported authentication mechanisms and I've seen some RetryLogic in the code base, but it would be much easier to implement with a few hints on how to do it.