utls
utls copied to clipboard
refraction-networking
Calling Read before Write ignores the fingerprint
Hello
If you call Read before Handshake or Write, the default go ciphers will be used. This is because the Write method is implemented for UConn and the handshake in it will call the utls handshake. But the Read method is implemented by Conn and executing it will cause the Read to call the tls handshake instead of utls one.
Is this intended? And if so, is there anyway (except Handshake) to establish the connection with fingerprints by calling Read at first?
Thanks
What's wrong with calling Handshake? This is an unusual use-case with client connecting, and waiting for server to send data.
On Thu, Apr 1, 2021, 01:56 Hirbod Behnam @.***> wrote:
Hello If you call Read before Handshake or Write, the default go ciphers will be used. This is because the Write method is implemented for UConn and the handshake in it will call the utls handshake. But the Read method is implemented by Conn and executing it will cause the Read to call the tls handshake instead of utls one. Is this intended? And if so, is there anyway (except Handshake) to establish the connection with fingerprints by calling Read at first? Thanks
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/refraction-networking/utls/issues/75, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJQ4S3SYCXN4IWXMNFNXBLTGQRLFANCNFSM42GRZLUQ .
Hello Yes indeed it is! I thought this might a bug in the code so I reported it. If this is intended, there is no problem with calling Handshake (which has been done in xray). Please close this issue if this is expected behavior.
We met the same problem while we tried to integrate utls into golang net/http package. As it creates two goroutine for reading and writing messages, there is no guarantee that Write() being called before Read(). Though it can be prevented by modifying application codes, I think UConn implements Read() which calls UConn.Handshake before calling super.Read() can easily solve it, without being confused by such behavior.
As it creates two goroutine for reading and writing messages, there is no guarantee that Write() being called before Read().
I was just bitten by this problem as well, in the same situation: a pair of read/write goroutines started immediately after calling UClient.
Calling Handshake before the first Read or Write is an effective workaround, but the silent failure to apply the chosen fingerprint is unexpected. I agree with @HirbodBehnam that it would be more natural if the connection works the same whether Read or Write is called first. Particularly as users of utls will be accustomed to the way Conn.Handshake works in crypto/tls; I read the documentation of Conn.Handshake and assumed utls would work the same way:
Most uses of this package need not call Handshake explicitly: the first Read or Write will call it automatically.
Here is a running example of the network communication pattern I was using. I do not think it is anything too unusual. You can run, for example, go run demo.go -- example.com:443 and then type GET / HTTP/1.0\n\n to get an HTTP response from the server—but the ClientHello will have the wrong TLS fingerprint. This is using go1.15.9 and utls v1.0.0.
demo.go
// Demo for https://github.com/refraction-networking/utls/issues/75
// go run demo.go -- example.com:443
package main
import (
"flag"
"fmt"
"io"
"net"
"os"
"sync"
"github.com/refraction-networking/utls"
)
func utlsDial(network, addr string, config *tls.Config) (*tls.UConn, error) {
conn, err := net.Dial(network, addr)
if err != nil {
return nil, err
}
uconn := tls.UClient(conn, config, tls.HelloChrome_83)
// err = uconn.Handshake()
// if err != nil {
// uconn.Close()
// return nil, err
// }
return uconn, nil
}
func demo(addr string) error {
serverName, _, err := net.SplitHostPort(addr)
if err != nil {
return err
}
tlsConfig := tls.Config{ServerName: serverName}
uconn, err := utlsDial("tcp", addr, &tlsConfig)
if err != nil {
return err
}
defer uconn.Close()
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
_, err := io.Copy(os.Stdout, uconn)
if err != nil {
fmt.Fprintf(os.Stderr, "stdout←uconn: %v\n", err)
}
}()
go func() {
defer wg.Done()
_, err := io.Copy(uconn, os.Stdin)
if err != nil {
fmt.Fprintf(os.Stderr, "uconn←stdin: %v\n", err)
}
}()
wg.Wait()
return nil
}
func main() {
flag.Parse()
if flag.NArg() != 1 {
fmt.Fprintf(os.Stderr, "%s: need a target host:port\n", os.Args[0])
os.Exit(1)
}
err := demo(flag.Arg(0))
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
Wireshark dissection of ClientHello
Transport Layer Security
TLSv1.3 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 255
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 251
Version: TLS 1.2 (0x0303)
Random: RANDOM
Session ID Length: 32
Session ID: ID
Cipher Suites Length: 38
Cipher Suites (19 suites)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
Cipher Suite: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
Cipher Suite: TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (0xc012)
Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)
Cipher Suite: TLS_AES_128_GCM_SHA256 (0x1301)
Cipher Suite: TLS_CHACHA20_POLY1305_SHA256 (0x1303)
Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302)
Compression Methods Length: 1
Compression Methods (1 method)
Compression Method: null (0)
Extensions Length: 140
Extension: server_name (len=15)
Type: server_name (0)
Length: 15
Server Name Indication extension
Server Name list length: 13
Server Name Type: host_name (0)
Server Name length: 10
Server Name: dns.google
Extension: status_request (len=5)
Type: status_request (5)
Length: 5
Certificate Status Type: OCSP (1)
Responder ID list Length: 0
Request Extensions Length: 0
Extension: supported_groups (len=10)
Type: supported_groups (10)
Length: 10
Supported Groups List Length: 8
Supported Groups (4 groups)
Supported Group: x25519 (0x001d)
Supported Group: secp256r1 (0x0017)
Supported Group: secp384r1 (0x0018)
Supported Group: secp521r1 (0x0019)
Extension: ec_point_formats (len=2)
Type: ec_point_formats (11)
Length: 2
EC point formats Length: 1
Elliptic curves point formats (1)
EC point format: uncompressed (0)
Extension: signature_algorithms (len=24)
Type: signature_algorithms (13)
Length: 24
Signature Hash Algorithms Length: 22
Signature Hash Algorithms (11 algorithms)
Signature Algorithm: rsa_pss_rsae_sha256 (0x0804)
Signature Hash Algorithm Hash: Unknown (8)
Signature Hash Algorithm Signature: Unknown (4)
Signature Algorithm: rsa_pss_rsae_sha384 (0x0805)
Signature Hash Algorithm Hash: Unknown (8)
Signature Hash Algorithm Signature: Unknown (5)
Signature Algorithm: rsa_pss_rsae_sha512 (0x0806)
Signature Hash Algorithm Hash: Unknown (8)
Signature Hash Algorithm Signature: Unknown (6)
Signature Algorithm: rsa_pkcs1_sha256 (0x0401)
Signature Hash Algorithm Hash: SHA256 (4)
Signature Hash Algorithm Signature: RSA (1)
Signature Algorithm: ecdsa_secp256r1_sha256 (0x0403)
Signature Hash Algorithm Hash: SHA256 (4)
Signature Hash Algorithm Signature: ECDSA (3)
Signature Algorithm: rsa_pkcs1_sha384 (0x0501)
Signature Hash Algorithm Hash: SHA384 (5)
Signature Hash Algorithm Signature: RSA (1)
Signature Algorithm: ecdsa_secp384r1_sha384 (0x0503)
Signature Hash Algorithm Hash: SHA384 (5)
Signature Hash Algorithm Signature: ECDSA (3)
Signature Algorithm: rsa_pkcs1_sha512 (0x0601)
Signature Hash Algorithm Hash: SHA512 (6)
Signature Hash Algorithm Signature: RSA (1)
Signature Algorithm: ecdsa_secp521r1_sha512 (0x0603)
Signature Hash Algorithm Hash: SHA512 (6)
Signature Hash Algorithm Signature: ECDSA (3)
Signature Algorithm: rsa_pkcs1_sha1 (0x0201)
Signature Hash Algorithm Hash: SHA1 (2)
Signature Hash Algorithm Signature: RSA (1)
Signature Algorithm: ecdsa_sha1 (0x0203)
Signature Hash Algorithm Hash: SHA1 (2)
Signature Hash Algorithm Signature: ECDSA (3)
Extension: renegotiation_info (len=1)
Type: renegotiation_info (65281)
Length: 1
Renegotiation Info extension
Renegotiation info extension length: 0
Extension: signed_certificate_timestamp (len=0)
Type: signed_certificate_timestamp (18)
Length: 0
Extension: supported_versions (len=9)
Type: supported_versions (43)
Length: 9
Supported Versions length: 8
Supported Version: TLS 1.3 (0x0304)
Supported Version: TLS 1.2 (0x0303)
Supported Version: TLS 1.1 (0x0302)
Supported Version: TLS 1.0 (0x0301)
Extension: key_share (len=38)
Type: key_share (51)
Length: 38
Key Share extension
Client Key Share Length: 36
Key Share Entry: Group: x25519, Key Exchange length: 32
Group: x25519 (29)
Key Exchange Length: 32
Key Exchange: EXCHANGE
Wireshark dissection of ClientHello, when Handshake is called
Transport Layer Security
TLSv1.3 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 512
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 508
Version: TLS 1.2 (0x0303)
Random: RANDOM
Session ID Length: 32
Session ID: ID
Cipher Suites Length: 32
Cipher Suites (16 suites)
Cipher Suite: Reserved (GREASE) (0x8a8a)
Cipher Suite: TLS_AES_128_GCM_SHA256 (0x1301)
Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302)
Cipher Suite: TLS_CHACHA20_POLY1305_SHA256 (0x1303)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)
Cipher Suite: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
Compression Methods Length: 1
Compression Methods (1 method)
Compression Method: null (0)
Extensions Length: 403
Extension: Reserved (GREASE) (len=0)
Type: Reserved (GREASE) (19018)
Length: 0
Data:
Extension: server_name (len=15)
Type: server_name (0)
Length: 15
Server Name Indication extension
Server Name list length: 13
Server Name Type: host_name (0)
Server Name length: 10
Server Name: dns.google
Extension: extended_master_secret (len=0)
Type: extended_master_secret (23)
Length: 0
Extension: renegotiation_info (len=1)
Type: renegotiation_info (65281)
Length: 1
Renegotiation Info extension
Renegotiation info extension length: 0
Extension: supported_groups (len=10)
Type: supported_groups (10)
Length: 10
Supported Groups List Length: 8
Supported Groups (4 groups)
Supported Group: Reserved (GREASE) (0x5a5a)
Supported Group: x25519 (0x001d)
Supported Group: secp256r1 (0x0017)
Supported Group: secp384r1 (0x0018)
Extension: ec_point_formats (len=2)
Type: ec_point_formats (11)
Length: 2
EC point formats Length: 1
Elliptic curves point formats (1)
EC point format: uncompressed (0)
Extension: session_ticket (len=0)
Type: session_ticket (35)
Length: 0
Data (0 bytes)
Extension: application_layer_protocol_negotiation (len=14)
Type: application_layer_protocol_negotiation (16)
Length: 14
ALPN Extension Length: 12
ALPN Protocol
ALPN string length: 2
ALPN Next Protocol: h2
ALPN string length: 8
ALPN Next Protocol: http/1.1
Extension: status_request (len=5)
Type: status_request (5)
Length: 5
Certificate Status Type: OCSP (1)
Responder ID list Length: 0
Request Extensions Length: 0
Extension: signature_algorithms (len=18)
Type: signature_algorithms (13)
Length: 18
Signature Hash Algorithms Length: 16
Signature Hash Algorithms (8 algorithms)
Signature Algorithm: ecdsa_secp256r1_sha256 (0x0403)
Signature Hash Algorithm Hash: SHA256 (4)
Signature Hash Algorithm Signature: ECDSA (3)
Signature Algorithm: rsa_pss_rsae_sha256 (0x0804)
Signature Hash Algorithm Hash: Unknown (8)
Signature Hash Algorithm Signature: Unknown (4)
Signature Algorithm: rsa_pkcs1_sha256 (0x0401)
Signature Hash Algorithm Hash: SHA256 (4)
Signature Hash Algorithm Signature: RSA (1)
Signature Algorithm: ecdsa_secp384r1_sha384 (0x0503)
Signature Hash Algorithm Hash: SHA384 (5)
Signature Hash Algorithm Signature: ECDSA (3)
Signature Algorithm: rsa_pss_rsae_sha384 (0x0805)
Signature Hash Algorithm Hash: Unknown (8)
Signature Hash Algorithm Signature: Unknown (5)
Signature Algorithm: rsa_pkcs1_sha384 (0x0501)
Signature Hash Algorithm Hash: SHA384 (5)
Signature Hash Algorithm Signature: RSA (1)
Signature Algorithm: rsa_pss_rsae_sha512 (0x0806)
Signature Hash Algorithm Hash: Unknown (8)
Signature Hash Algorithm Signature: Unknown (6)
Signature Algorithm: rsa_pkcs1_sha512 (0x0601)
Signature Hash Algorithm Hash: SHA512 (6)
Signature Hash Algorithm Signature: RSA (1)
Extension: signed_certificate_timestamp (len=0)
Type: signed_certificate_timestamp (18)
Length: 0
Extension: key_share (len=43)
Type: key_share (51)
Length: 43
Key Share extension
Client Key Share Length: 41
Key Share Entry: Group: Reserved (GREASE), Key Exchange length: 1
Group: Reserved (GREASE) (23130)
Key Exchange Length: 1
Key Exchange: 00
Key Share Entry: Group: x25519, Key Exchange length: 32
Group: x25519 (29)
Key Exchange Length: 32
Key Exchange: EXCHANGE
Extension: psk_key_exchange_modes (len=2)
Type: psk_key_exchange_modes (45)
Length: 2
PSK Key Exchange Modes Length: 1
PSK Key Exchange Mode: PSK with (EC)DHE key establishment (psk_dhe_ke) (1)
Extension: supported_versions (len=11)
Type: supported_versions (43)
Length: 11
Supported Versions length: 10
Supported Version: Unknown (0x7a7a)
Supported Version: TLS 1.3 (0x0304)
Supported Version: TLS 1.2 (0x0303)
Supported Version: TLS 1.1 (0x0302)
Supported Version: TLS 1.0 (0x0301)
Extension: compress_certificate (len=3)
Type: compress_certificate (27)
Length: 3
Algorithms Length: 2
Algorithm: brotli (2)
Extension: Reserved (GREASE) (len=1)
Type: Reserved (GREASE) (51914)
Length: 1
Data: 00
Extension: padding (len=210)
Type: padding (21)
Length: 210
Padding Data: 000000000000000000000000000000000000000000000000000000000000000000000000…
I think the root cause of this issue finally became a problem (see #284). A new issue is opened as #291 and we have a PR (#292) ready for this.
I know this fix is kinda late, but let's hope in the future such thing can be patched in a timely manner.