Signal's TLS Proxy Failed to be Probing Resistant and seems leaky

[DuckSoft]

Links

• Signal Post: https://signal.org/blog/help-iran-reconnect/
• Their Repository: https://github.com/signalapp/Signal-TLS-Proxy
• Our Original Issue: https://github.com/signalapp/Signal-TLS-Proxy/issues/3

Why Here?

:warning: emotional. don't read.

So I've studied this with @studentmain and found it problematic about 4 hours ago. We immediately reported this through a GitHub Issue, with PoC and advice attached, without even sleeping (it's about 4:00 am in local time) But @moxie-signal (@moxie0) from @signalapp just closed our issue, saying this:

Hey everyone! Thanks for the interest in this. We normally don't use GH issues for this type of discussion, though, and prefer to have that happen over on the Signal community forum. Please check out https://community.signalusers.org/ if you want to get involved. Thanks!

It seemed that they just don't care about probing resistant and simply shut people's mouths up instead of fix this issue.

And guess what's next? THEY THEN JUST CLOSED THE WHOLE ISSUE FUNCTION!

THEY CLAIMED TO HELP PEOPLE IN CENSORSHIP, BUT THEY IN TURN CENSOR WHISTLEBLOWERS.

And here's what has happened if I go to "Signal community forum" to post the issue: Okay, fine. You guys have really an awesome security!

Some updates: They even banned me from their GitHub organisation! What an honour!

---

Extra notes for those guys who think I was aggressive and not considerate enough:

You guys are correct. I ain't saint. I got my emotions and temper. The weapon of criticism obviously cannot replace the criticism of weapons. Material force must be overthrown by material force. But theory also becomes a material force once it has gripped the masses. So long as they were ignoring the weapon of criticism, I think it fair to put some criticism of weapons -- Or I can completely ignore everything: Don't post it at all, and grab some popcorn to watch -- Just wild guessing, Iranian being caught using Signal and sent to jail?

---

Updates:

So that's why I'll post it here.

Original Issue

Issue Body

I am not a specialist on Censorship Circumvention, so if I'm wrong, I'd appreciate if I could be corrected.

I've quickly scanned the code, so this is just a TLS tunnel set up to forward inner TLS traffic with signal SNIs as-is. But what if the censor actively probe the suspected proxies? For example, connect and then use real signal app traffic and non-signal app traffic? The difference can be detected. And if a proxy can be detected, it can be blocked.

Thanks my friend @studentmain for the PoC.

PoC from @studentmain

PoC Source Code:

package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net"
	"os"
	"time"
)

func send(addr, server, sni string) int {
	c0, e := net.Dial("tcp", addr)
	if e != nil {
		log.Fatal(e)
	}

	c1 := tls.Client(c0, &tls.Config{
		ServerName:         server,
		InsecureSkipVerify: true,
	})

	c2 := tls.Client(c1, &tls.Config{
		ServerName:         sni,
		InsecureSkipVerify: true,
	})
	c2.SetDeadline(time.Now().Add(2 * time.Minute))
	s := fmt.Sprintf("GET / HTTP/1.1\r\nHost: %s\r\nUser-Agent: curl/7.68.0\r\n\r\n", sni)
	//b := make([]byte, 4096)
	l, _ := c2.Write([]byte(s))
	log.Println(l)
	if e != nil {
		return 0
	}
	log.Printf("%s->%s->%s\n", addr, server, sni)
	return l
}
func main() {
	if len(os.Args) != 3 {
		log.Fatalln("usage: main.exe server_name addr_port")
	}
	server := os.Args[1]
	addr := os.Args[2]
	l1 := send(addr, server, "updates.signal.org")
	l2 := send(addr, server, "telegram.org")
	if l1 != 0 && l2 == 0 {
		log.Fatalln("It is a Signal TLS Proxy")
	}
	log.Println("It is not a Signal TLS Proxy")
}

PoC Example Run from @studentmain

Here is an example run of the PoC from @studentmain:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

C:\src\signalTlsPoc>main.exe
2021/02/05 03:40:15 usage: main.exe server_name addr_port

C:\src\signalTlsPoc>main.exe your.proxy.server.example.com your.server.ip.address:443
2021/02/05 03:40:34 69
2021/02/05 03:40:34 your.server.ip.address:443->your.proxy.server.example.com->updates.signal.org
2021/02/05 03:40:35 0
2021/02/05 03:40:35 your.server.ip.address:443->your.proxy.server.example.com->telegram.org
2021/02/05 03:40:35 It is a Signal TLS Proxy

C:\src\signalTlsPoc>main.exe baidu.com 39.156.69.79:443
2021/02/05 03:43:09 0
2021/02/05 03:43:09 39.156.69.79:443->baidu.com->updates.signal.org
2021/02/05 03:43:10 0
2021/02/05 03:43:10 39.156.69.79:443->baidu.com->telegram.org
2021/02/05 03:43:10 It is not a Signal TLS Proxy

C:\src\signalTlsPoc>

(modified to protect my personal information)
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEdbyjO018JNZhjd2SqnhRnCCMh0IFAmAcT0AACgkQqnhRnCCM
h0Lx6RAAlx6C6R7tZROlj8I+1uu2Owde9EjOA3BTYw7aC8pr4cd2LuTkmAY07TuZ
mbQRueUIKOXx2ZlTPRL1yOAW60nePC5SRmlOkNssafWzM+8zyqfHdRe64OX0ZA5V
KPgSVHOz2JKfh650oEqlaHb/6N/Nj++617TcHmmYoy/xsiDy/fD7IoifD8bT5N3+
4znZ2Sr2+IirKxqvpVDl5AwRBR5EfXUI54c0OjKB8Sq+VPlhAmAcj/eZgSz7A4rV
EWSJjwbN1fp7g2C4jNtjh0Z3jyvvA9yGNThylcMPbXcrc1a9sOzhbCgGTtF1413n
tXSjy3vRBEnwvlVs78ItQljCZV1S+jwpXlkjFPGAIYyRRQbvaQ4Biaki/4g3ofVu
nfqEnc5m4vSsde2r5GLBqxSkbsKuZjGBbd1KRE+5LZPvkVrA5wP400tirThnvNU9
G7S5Noz/8Npwazag0beDlio45e8sBafTBRJy/xuq6HVGdFkPa1C7EUMuDxVVnPdg
mOgyqrjXju+4y/pl39U6JmDt6/ZSn34/1EFVchRWS5Kq0gPDGdjvrjI88aOtqhRH
tT8F6dU8rwRa8tjMHv3gNVTQY+jQZIi4SHNB54DRc1K0AKzIkAOgD/ROhcU4LkXq
vudKRTfhbtL6l8FIo8WCzGVONK1dY+1alKXKc8cVJNrye/hm5t4=
=H1r/
-----END PGP SIGNATURE-----

Suggestion 1 from @ducksoft

One crude idea is to use preshared tokens and fallback mechanism.

~~In this case, we can hide a preshared key within SNI. For example, we can preshare a key 538fd09a066a, and then mix this with original SNIs, where cdn.signal.org could become 538fd09a066a.cdn.signal.org. We then only accept whitelisted SNIs mixed with our preshared key, and deny all other SNIs including the original one cdn.signal.org. This way without knowing the preshared key, the censor is unable to active probe any longer. Since the SNI is actually sent inside the TLS tunnel after establishing a TLS connection with the proxy (TLS over TLS), the inside SNI is virtually invisible to the censors (unless root certificates got hijacked, etc.).~~(Edited: this is only feasible when those domains are transparent about SNIs and hosts, thus not operatable) And, since the tunnel is over TLS, you can virtually hide the preshared key anywhere inside, even making this as a customized TLS ClientHello Extension, or just masquerade as a plain HTTP request but with special headers, etc.

It is also possible to utilize WebSocket over TLS on the outside, so that unless the exact WebSocket endpoint path is known (which also works as a preshared key), the proxy cannot be probed by the censors without knowing the path - maybe they can only see a sad 404 page or your blog index page. But one thing must be clear: From the ClientHello ALPN extension, censors can deduce that you are using WebSocket techniques, since your ALPN cannot be h2/http/1.1 in this case, and WebSocket don't work when there's a h2 connection. You can turn off the h2 support on the server side and proceed in this situation, but normal browsers don't tend to connect WebSocket just all the time and they will prefer the use of h2 unless clearly denied.

For the fallback mechanism, when the traffic is found invalid, we should act the same as those innocent servers. When innocent servers is sent with TLS handshake inside TLS tunnel, the server tends to disconnect with bad request response. Then we should act the same. And if a valid HTTP request is received, we can then fallback to an innocent website, such as blog, minigames or something. This will significantly increase the difficulty for the censors to block.

Suggestion 2 from @ducksoft

Also there's a weak temporal feature in this project, where the certificates tend to be newly signed. If possible, relay hosts should try to reuse existing certificates rather than brand new ones, which can be far more suspicious to the censors.

Suggestion 3 from @ducksoft

Besides, for large portions of TLS traffic, there would be an ALPN in the Client Hello frame. If this proxy runs TLS over TLS, then the ALPN would possibly not be commonly seen ones like h2 and http/1.1. Instead, the application tend not to send this extension, which creates a passive feature of this proxy.

But what if we pretend to be h2/http/1.1? Well, you can actually do it, though you are breaking the rules. I am not sure about the exact behavior, but this could create problems when you want to be compatible with existing web servers.

Suggestion 4 from @ducksoft

By the way, DPI (Deep Packet Inspection) techniques can detect TLS in TLS traffics easily (there are lots of papers available, if you do a search). Although not 100% TLS in TLS traffic is our lovely proxy, you can still risk being caught using a proxy when censorship is there.

DPI might be expensive, but code like @studentmain is not. The current status of probing resistance is really a problem.

Archive

⚠️_Failure_to_be_Probing_Resistant_with_PoC_·Issue#3_·_signalapp.zip

What Leaky?

https://community.signalusers.org/t/traffic-not-routed-to-tls-proxies-can-expose-users-to-censors/27479

[wkrp]

Thanks for making this post. To me it looks like your analysis is correct: a countermeasure to this kind of proxy is to watch for outgoing TLS connections and then connect to the same TLS servers, sending a tunneled TLS stream for one of the signal.org domains. The proof-of-concept prober works for me, using example.com and a proxy I found posted online. I know it's frustrating to have your bug reports deleted, but I would ask let's please try to be considerate and constructive. We are all on the same side.

A TLS-in-TLS tunnel with restricted endpoints is not a bad design for circumvention, in my opinion. It hides the endpoint address while protecting the client's end-to-end security. You are correct, though, that to prevent active probing, the proxy must require the client to prove knowledge of some secret. A simple way to do it is to have the client send a password or token when it connects, like NaïveProxy does with probe-resistant HTTP authentication, HTTPT does with a URL path, and Rosen does with an HTTP header. The key idea is that there must be more information required to use the proxy than just the IP address / domain name, because those pieces of information are visible to a censor.

I had a look at the Signal-TLS-Proxy repo, though, and it's not a custom piece of software: it's a Docker configuration that installs two instances of Nginx, an internal instance that handles the forwarding to signal.org servers, and an external instance that terminates the outer layer of TLS and forwards to the internal instance. It's actually a pretty clever way of quickly constructing a proxy that's easy to install. But it means that one cannot just hack in a custom client authentication scheme. I am not too familiar with Nginx configuration files, but @DuckSoft's idea of a secret subdomain prefix may be workable, if there's a way to strip the subdomains before forwarding. (Edit: I realized later that the idea of stripping subdomains from SNI is nonsense, as end-to-end integrity checks would prevent it. I was concerned about the secret serving as a persistent and externally visible user identifier, but as long as all users of a proxy use the same secret, it's not any more identifying than the proxy IP address.) Something like this would increase the complexity of deployment, though, as every proxy would need to have a unique nginx-relay/nginx.conf file, and there would need to be a way to share the secret as part of a signal.tube or sgnl:// link.

I think HTTPT's faux WebSocket forwarding design is clever. It enables the external TLS layer to be HTTPS, which provides places to communicate a secret. The payload does not actually need to be WebSocket—the external HTTPS server just forwards the contents without actually trying to interpret them as the WebSocket protocol. But it still requires a custom server program to absorb the forwarded HTTP headers and then proxy the remainder of the connection. It's definitely not as simple as an Nginx proxy.

[proletarius101]

If I didn't make it wrong, we are just trying to reinvent https://www.v2ray.com/. Maybe a custom variant of v2ray is a solution, as we are using docker. It even has built-in sni sniffing.

[DuckSoft]

A friend of mine used his own Signal Community account to repost this issue: https://community.signalusers.org/t/tls-proxy-server-unable-to-survive-active-probing-from-internet-surveillance-systems/27282

Till now (2021-02-05 03:50:59 UTC) I've got no replies, none from any staff.

---

Updates: Okay, at least we got the first one:

It seems like a valid concern, but I’m not sure why it would be a reason to choose to not run a proxy.

I wish I could comment, but my account is not able to post messages on that forum.

[sergeyfrolov]

Back when we were working on The use of TLS in Censorship Circumvention paper, we found some issues in TLS ClientHello messages, generated by Signal client. You can find more info in the paper. We sent the signal team a couple e-mails, privately informing them about our findings. After getting no response in a couple months, we submitted a github issue. Here it is: https://github.com/signalapp/Signal-Android/issues/7337

So, I understand how you feel about the issue, and it is unreasonable that Signal suspended your forum account after telling you to create a post there, but I must agree with @wkrp reminder about civility: we are on the same side. Calling their approach stupid on their forum and naming PoC github repo "fuck-signal-tls-proxy" doesn't help either. If Signal does bury their heads into the sand and does not address the issue, let's just grab some popcorn and see how long it takes them to get blocked.

[DuckSoft]

@sergeyfrolov Well, I got to clarify something first.

1. The repo is composed and named by @studentmain, where I respect the freedom of his naming his own project;
2. I am not the very author who've put "stupid" in their forum, and back then my account was suspended. You will see the post was by Teleposter rather than DuckSoft.

Anyway, let's see what will be on. Thanks for your reply.

[wkrp]

I like to try to set an encouraging tone. It is, of course, fair play to point out security vulnerabilities, but let's also give credit where it is due. The blog post says Signal deployed this new system in about 10 days, which is a remarkably fast response. It calls the new simple TLS proxy an "interim solution" and says "we're also continuing to investigate other techniques that are more automated and convenient." So far as that's true, the simple TLS proxy may be good enough, for now, to deal with the blocking in Iran. If the censors there do not already have an active probing system set up (and I don't know of any evidence that they do), they won't be able to create one instantly. (It would be a different story in China, of course, because the GFW's existing active probing of Tor and Shadowsocks, among other protocols, could probably easily be adapted for Signal proxies.)

@DuckSoft and @studentmain also deserve credit for analyzing the proxy and finding the flaw so quickly. Not everyone has the knowledge and skill to do something like that. I value your participation on this forum, and your help in joining together different groups of circumvention developers. People like us are accustomed to recognizing common pitfalls such as susceptibility to active probing. We gained that knowledge through experience, and it's the same for any other developer. Let's continue to look for flaws while also encouraging and supporting our colleagues who have good intentions.

[ghost]

they won't be able to create one instantly

Time is money. At the moment, our only chance (not only for Signal, I mean everyone) is develop and deploy new solution before they deploy new system. Luckily, small teams can response way faster than a huge organization. If it works now, then it works now, just use it. Then let's build something which works in the darker future. And knowledge is power. If they know the sad stories in this forum at the beginning, they can just learn from it, avoid these problem without try and error.

[ghost]

Perdí la confianza en esta aplicación, lista negra +1

---

Translation added by @wkrp: I lost confidence in this application, blacklist +1

[luckydonald]

To bad the issue was never crawled to archive.org in the original repository. That makes it sadly difficult to verify the statement of actually making them aware.

[DuckSoft]

To bad the issue was never crawled to archive.org in the original repository.

That makes it sadly difficult to verify the statement of actually making them aware.

No if you use GitHub App on iOS, the issue is still visible.

[DuckSoft]

To bad the issue was never crawled to archive.org in the original repository. That makes it sadly difficult to verify the statement of actually making them aware.

Also you can go to GitHub GraphQL Explorer and perform the following query, where you can see the original post and responses:

query { 
  repository(
    name:"Signal-TLS-Proxy", 
    owner:"signalapp"
  ) {
    issue(number: 3) {
     	author {
        login
      }
      body
      comments(first: 9) {
        nodes {
          author {
            login
          }
          body
        }
      }
    }
  }
}

[exabrial]

There are so many better ways to make important points without resorting to personal attacks, ultimatums, or veiled threats. I think that should be talked about.

Take this sentence:

It seemed that they just don't care about probing resistant [sic] and simply shut people's mouths up instead of fix this issue.

Even if it were on purpose, why not consider a de-escalation first; why go on the offensive? Undoubtedly if you are passionate about something, eventually you will become frustrated. It's part of growth and progress, but part of that two is not giving in to feelings of frustration, anger, or rejection.

I think there are two things that could be applied here that resulted in the missed high five.

• First: Seek first to understand, then to be understood. Essentially, seek to understand why they closed the ticket, rather than reacting as if it were an attack. https://en.wikipedia.org/wiki/The_7_Habits_of_Highly_Effective_People#5_-_Seek_first_to_understand,_then_to_be_understood

• Second: Assume positive intent. When someone does something unexpected or unpredicted, assume they actually were acting with good intentions, and that it was not an attack on you. This is highly related to the first point. In the The 5 Actions of Smart Trust they talk about this and I'm always reminded of that when dealing with over-worked devs on open source projects.

I hope this all shakes out for the positive. I'm very thankful for the Signal team and the people that take their time to audit the security and submit bugs.

[benlivengood]

Don't put secrets in SNI or any other plaintext before the TLS session is established with the proxy. Anything in plaintext can be replayed by an adversary.

[wkrp]

Don't put secrets in SNI or any other plaintext before the TLS session is established with the proxy. Anything in plaintext can be replayed by an adversary.

I think you misunderstand the proposal. There are two layers of TLS in the tunnel: the outer layer that terminates at the simple TLS proxy, and the inner layer that is end-to-end between the client and signal.org servers. The proposal is to include an authentication secret with the inner layer of TLS, where it is invisible to an adversary. The outer TLS layer would remain as normal. It's a sound model, used by many existing and proposed circumvention systems, including those mentioned in https://github.com/net4people/bbs/issues/60#issuecomment-773748535. There are many ways to do it, but I believe in this thread we were brainstorming ways that would fit easily with the Docker setup.

[ghost]

in this thread we were brainstorming ways that would fit easily with the Docker setup.

Proxy in Docker is not something new, you know. Outline is the most famous one. I'm thinking about is there something only they can do? Like, use their already existed account system, protocol, etc, to provide authentication support, detect bad users, etc.

[petermolnar]

The more I read the more this whole setup resembles port forwarding over SSH, and via connecting TLS over the tunnel.

[wkrp]

The more I read the more this whole setup resembles port forwarding over SSH, and via connecting TLS over the tunnel.

In the sense of tunneling one protocol in another, yes. SSH is probably an inferior cover protocol (less costly to block) than TLS, though.

The naive and insecure way of constructing a TLS proxy is to have the client's TLS session terminate at the proxy, and the proxy re-encrypt the stream in a separate TLS session to the destination. The problem with that is that the proxy may inspect and tamper with the traffic between the client and the destination. Signal's simple TLS proxy avoids that pitfall with its TLS-in-TLS design. The outer layer to the proxy is for circumvention only: it hides the SNI and other destination identifiers such as the destination IP address. The inner layer, which the proxy cannot tamper with, is for end-to-end encryption and authentication with the Signal servers.

[DuckSoft]

What an Honour! @signalapp just blocked me!

and @studentmain!

I think it fair to put here now:

[abschluss24]

We sent the signal team a couple e-mails, privately informing them about our findings. After getting no response in a couple months, we submitted a github issue. Here it is: https://github.com/signalapp/Signal-Android/issues/7337

I just got a page not found (archive) by visiting this link. No idea what was wrong.

If the issue @sergeyfrolov opened was just deleted (archive.org keeps only one archive of that page so I cannot be sure), then it implies that the Signal team is watching this thread, without actively participating in the conversation.

[DuckSoft]

We sent the signal team a couple e-mails, privately informing them about our findings. After getting no response in a couple months, we submitted a github issue. Here it is: https://github.com/signalapp/Signal-Android/issues/7337

I just got a page not found (archive) by visiting this link. No idea what was wrong.

@abschluss24 It's surely deleted, my friend. Signal just do ostriches and refuse to handle the problem, which is pretty sad.

[ghost]

That's true.

I can visit both https://github.com/signalapp/Signal-Android/issues/7336 and https://github.com/signalapp/Signal-Android/issues/7338

Since GitHub PR and Issues have an self-increment unique ID, the only reason to the 404 https://github.com/signalapp/Signal-Android/issues/7337 would be the post deletion.

Update: See https://github.com/signalapp/Signal-Desktop/issues/4513

[DuckSoft]

the Signal team is watching this thread, without actively participating in the conversation.

Maybe, yeah. I just don't understand why they don't process the problem itself, rather than "process" us whistleblowers.

[ghost]

Hi, Signal team, if you are watching this, thank you. Keep watching, and reading other issues, learn from them, then update your code, create a perfect solution for your user, you can do it.

[ghost]

as far as I know

If can’t solve the problem, then "solve" the one who raised the problem

This is the style of the Chinese government

Yeah, i got it

[ghost]

as far as I know

If can’t solve the problem, then "solve" the one who raised the problem

This is the style of the Chinese government

Yeah, i got it

Is there a possibility that this is not a bug its just a pre-preserved backdoor?

[DuckSoft]

Is there a possibility that this is not a bug its just a pre-preserved backdoor?

I'd try my best to resist resorting to conspiracy, but... I don't know. I just felt sad.

[ghost]

as far as I know If can’t solve the problem, then "solve" the one who raised the problem This is the style of the Chinese government Yeah, i got it

Is there a possibility that this is not a bug its just a pre-preserved backdoor?

No conspiracy theory, please.

[Equim-chan]

To develop an anti-censorship tool, you must have an idea of what you are going to achieve. The most fundamental objective for an internet censorship circumvention method to achieve is the resistance to detection against an active probing adversary. Such requirement overlaps with steganography more than cryptography. Cryptanalysis inspects data that is known to contain a message, while steganalysis inspects whether a piece of data contains any hidden message. On the other hand, the demand for the detection resistance is so critical that even the encryption is not necessary, say, if you manage to exploit the vulnerability of the censorship system itself[1][2].

[1] scholarzhang. 2010. West Chamber Project. (2010). Retrieved February 7, 2021, from https://code.google.com/archive/p/scholarzhang [2] Zhongjie Wang, Yue Cao, Zhiyun Qian, Chengyu Song, and Srikanth V. Krishnamurthy. 2017. Your state is not mine: a closer look at evading stateful internet censorship. In Proceedings of the 2017 Internet Measurement Conference (IMC '17). Association for Computing Machinery, New York, NY, USA, 114–127. DOI:https://doi.org/10.1145/3131365.3131374

[DuckSoft]

If you are designing a system whose functions include providing evidence, it had better be able to withstand hostile review. - Ross J. Anderson

[ghost]

Update of PoC:

I added a packet capture library (github.com/google/gopacket) into my PoC, now I can experiment with the passive detection part. So, I wrote a naive TLS fingerprint checker to find possible Signal TLS Proxy connection.

https://github.com/studentmain/fuck-signal-tls-proxy/blob/5445131a02ef7cf1fb865b3e3c2c960390d1fd56/poc.go#L130-L147

It's very inaccurate at the moment, even reports a false positive to my IDE's HTTP client. But it reports negative to browser, that's enough. Then we can sending probe to possible proxy to filter out false positive. You already see how it works in the original PoC.

For non-proxy Signal connection, they have a plaintext SNI (CDN disallow domain fronting, so we can only waiting for ECH at the moment, it's not a problem for Signal, nobody can resolve it now), so detecting them is as easy as string.EndsWith()...

https://github.com/studentmain/fuck-signal-tls-proxy/blob/5445131a02ef7cf1fb865b3e3c2c960390d1fd56/poc.go#L149-L163

Maybe I can improve TLS fingerprint checker's accuracy by checking corresponding ServerHello or just check more detail in ClientHello?