mtprotoproxy icon indicating copy to clipboard operation
mtprotoproxy copied to clipboard

several TLS records

Open zhdkirill opened this issue 5 years ago • 16 comments

I'm getting the following message in log for every domain I try: The MASK_HOST $host returned several TLS records, this is not supported What am I doing wrong? What domain can I use that is supported? Even my own domains with TLS1.3 only do not fit.

zhdkirill avatar Nov 20 '19 16:11 zhdkirill

OK, so there are 2 issues.

  1. Why if certificate length is less than MIN_CERT_LEN it marked as "several records"? It is more like insufficient record length, and I'm not quite getting why is it bad. And it's definitely one record, not several.
  2. How can I get my TLS record of sufficient length? Is it possible with Let's Encrypt certificates?

So far I found that google.com and cloudflare.com are good enough, but making Fake-TLS with my actual domain would be much better in terms of obscurity.

zhdkirill avatar Nov 20 '19 18:11 zhdkirill

The MASK_HOST www.ya.ru returned several TLS records, this is not supported

И еще одна интересная: Failed to connect to MASK_HOST www.ya.ru: Multiple exceptions: PySocks doesn't support IPv6: ('2a02:6b8::2:242', 443, 0, 0), Socket error: 0x01: General SOCKS server failure

pasha-zzz avatar Nov 22 '19 13:11 pasha-zzz

this message is about tls records, so the server is likely returns two tls records:

  1. the encrypted extensions record
  2. the record with the certificate

The encrypted extensions field are sent by most https servers and the telegram client are not able to parse such answers. The official proxy also refused to use www.ya.com with this message: Failed to parse upstream TLS response: TLS <= 1.2: expected x25519 as a chosen cipher

I'll try to find a way to make the client to eat the answers of this structure, but I am not sure that this is possible without the telegram client modification. For now the solution is to try more sites.

alexbers avatar Nov 22 '19 15:11 alexbers

https://www.ssllabs.com/ssltest/analyze.html?d=www.ya.ru&s=87.250.250.242&latest

3 certs: yandex.az, Yandex CA and Certum

pasha-zzz avatar Nov 23 '19 06:11 pasha-zzz

The problem is not in the certificate number, but in the way how the tls protocol works. The tls can work two different ways - to send the certificate first, and then extensions or vice versa. The Telegram client support only the first scheme, so it would be easy to detect in traffic on the host who use the second scheme because in this case the proxy should answers by the first scheme for the real telegram clients and by the second scheme to the fake ones (i.e. fingerprinters).

But there are huge amount sites which work ok with the proxy. For example all hosts from cloudflare or most google subdomains.

alexbers avatar Nov 26 '19 14:11 alexbers

Не понятно как решить данную проблему, если использовать собственный домен. В моем случае схема следующая: haproxy -> mtproto -> caddy. До 1.0.8 эта схема работала корректно, сейчас я также получаю данную ошибку.

alexanderek avatar Dec 10 '19 18:12 alexanderek

So is this going to be fixed, or should be closed?

HosseyNJF avatar Apr 17 '20 15:04 HosseyNJF

It is fixable only by the client side, so the clients have to be patched

alexbers avatar Apr 20 '20 16:04 alexbers

It is fixable only by the client side, so the clients have to be patched

I am also getting the same error. So how it should be fixed?

xenstar avatar Apr 27 '20 22:04 xenstar

@alexbers hi! just teseted with TLS_DOMAIN alexbers.com

The MASK_HOST alexbers.com returned several TLS records, this is not supported Printed len(cert) to output, got 42.

For my own host I've got 36, and for ya.ru - 32 in the output.

Looks like a bug while getting the cert length. Only Google works, prints 4149. If that isn't a bug then I want to figure out what's the real issue is.

Python 3.9.15

aepot avatar Oct 26 '22 10:10 aepot

hi, please check it your site works using tls 1.3 because telegram masks as this tls version.

alexbers avatar Oct 26 '22 11:10 alexbers

@alexbers here's a dump of TLS handshakes

Format:

record_type
record_length
tls_version

My host TLS handshake:

22
122
0x0303
20
1
0x0303
23
36 <-- I have no idea what's inside the record body, looks like random bytes
0x0303
23
4031 <-- cert length
0x0303

It's definetely 1.3 but the real cert length is in 4th TLS Record not in 3rd as in Google's handshake.

Here's Google handshake:

22
122
0x0303
20
1
0x0303
23
4148 <-- cert length
0x0303

I understand that there's really multiple TLS Records but it's still possible to get Certificate even from my host or yandex, even from alexbers.com

UPD: totally disabled TLS 1.2 in nginx site config but no effect, the output is the same.

UPD2: applied an ugly patch

async def get_encrypted_cert(host, port, server_name):
    async def get_tls_record(reader):
        try:
            record_type = (await reader.readexactly(1))[0]
            tls_version = await reader.readexactly(2)
            if tls_version != b"\x03\x03":
                return 0, b""
            record_len = int.from_bytes(await reader.readexactly(2), "big")
            record = await reader.readexactly(record_len)

            return record_type, record
        except asyncio.IncompleteReadError:
            return 0, b""

    reader, writer = await asyncio.open_connection(host, port)
    writer.write(gen_tls_client_hello_msg(server_name))
    await writer.drain()

    record1_type, record1 = await get_tls_record(reader)
    if record1_type != 22:
        return b""

    record2_type, record2 = await get_tls_record(reader)
    if record2_type != 20:
        return b""

    record3_type, record3 = await get_tls_record(reader)
    if record3_type != 23:
        return b""

+    if len(record3) < MIN_CERT_LEN:
+        record4_type, record4 = await get_tls_record(reader)
+        if record4_type == 23 and len(record4) >= MIN_CERT_LEN:
+            return record4

    return record3

Now it works well with my host as TLS_DOMAIN in both TLS and Secure modes at least with official PC and Android clients.

Output: Got cert from the MASK_HOST $host, its length is 2659

aepot avatar Oct 26 '22 12:10 aepot

@alexbers Looks like the above patch works like a charm for last couple of months. Is it possible to make a better/proper fix?

Sorry I'm not a dev but admin and know nothing about python at all. Without patch it doesn't work with anything except Google as MASK_HOST.

On other hand if I'm wrong and that's unacceptable solution, then I have no idea how to setup my nginx to be a proper MASK_HOST that works with original mtproxy.

aepot avatar Dec 08 '22 23:12 aepot

@aepot Maybe you'd like to just send it as a pull request and see if it can get merged?

PeterDaveHello avatar Feb 24 '23 13:02 PeterDaveHello

should be fixed by https://github.com/alexbers/mtprotoproxy/commit/51b2482dec4de5135b562d44bd9ae18ef5d283a7

But this is makes the proxy more detectable, since there will be difference between handshake with real domain and handshake with proxy-server. As I told previously, the handshake with proxy server can't be changed without telegram client modification. But now at least proper certificate length will be used, which is slightly better then the random length.

alexbers avatar Feb 26 '23 19:02 alexbers

Looks like it works. Great thanks! Probably the issue can be closed.

Strange but before today it worked only with google.com as MASK_HOST but not with any other HTTPS server. I didn't find any clue how to setup my nginx to send the certificate as expected by MTProxy.

aepot avatar Feb 26 '23 21:02 aepot