probe
probe copied to clipboard
ooni
New experiment: measure port based filtering
This test would require a test helper that listens on all ports which we intend to measure for filtering.
The probe would then attempt to connect to every port on this target host and discover if it's able to establish a connection.
I think this would be very useful to uncover basic forms of censorship that just rely on simpler port filtering. It would also have the side-effect of being able to discover if some ports are reachable during an internet outage event (see for example open ports in KZ during the blocking event of 2022: https://ntc.party/t/network-shutdown-all-around-kazakhstan/1601/20).
@hellais I would imagine it would require a list of ports that are expected to work instead of checking each and every port. I wonder if it doesn't range too much as we increase the number of ip addresses in the pool. Even if we were to check all the tens of thousands of ports then maybe we can expect concurrency to be on our side but not for long.
@SrijanSriv as a starting point, I think we should be considering the most popular ports, i.e., 22/tcp, 23/tcp, 25/tcp, 80/tcp, 143/tcp, 443/tcp, 445/tcp, 587/tcp, and 993/tcp.
If you'd like to take on the task of writing this experiment, I recommend copying from the internal/engine/example experiment to create a new experiment. For performing the TCP connect proper, you should probably be using the primitive named TCPConnect inside the internal/measurex package. Once that is done, we should then probably discuss whether we'd like to use the same data format of the original tcp_connect experiment (see https://github.com/ooni/spec/blob/master/nettests/ts-008-tcp-connect.md) or use a different data format. In designing this experiment, we should also consider the extent to which we want to allow the input to be a domain name + port as opposed to being an IP address to port, which also we can discuss at a later stage, I guess.
Lastly, I would recommend taking a look at the ./internal/tutorial/measurex package because I am quite sure there is code in there that can be useful as a blueprint for implementing this functionality.
@bassosimone so I noticed that some of the experiments present in internal/engine/experiment try to implement ports but usually when we provide it to them (see riseupvpn). A few of them ( like fbmessenger, sniblocking and telegram) rely on just 443/tcp. Not all of these implementations have a certain uniformity (i.e some are using constants with the port included and some are adding the port in the end) which would mean that it would be better to change these by themselves (so for each test, in their own file). The tests could also be changed a bit so they can all call a certain array of ports when required.
More importantly, I don't get how dnscheck works around a port. Specifically, I don't know how, for a dns that is provided, ParseIP ever returns something not equal to nil. Does that mean URL is not required, just addr? Not to mention that addr doesn't provide us with a port (at least for now, by checking the test).
E: I now realize that makeResovlerURL is actually called after LookupHost is called, hence URL provided is meant to be an address anyways.
Yeah how I was imagining a basic version of this is that we would have 2 components:
- A test helper that is just a simple TCP service that listens on all ports we care to measure
- A nettest that does a simple TCPConnect to the test helper checking if it's able to establish a connection to the target endpoint
In terms of which ports we should be probing, I think a good starting point would be to look at the nmap-services file that gives an "open-frequency" to each port, making it a reasonable starting point.
If we wanted to make this test more generic and adaptive, we could also imagine having a control protocol between the test helper and the probe, where the test-helper tells the probe which ports it should be checking.
Going even further, we could imagine having the test-helper serve the probe a sort of FSM of the client <-> server message exchanges to reproduce traffic that looks like the traffic of the target port (as an example of this approach see the glasnost DSL: https://github.com/marcelscode/glasnost/blob/master/src/protocols.spec). This would allow us to also use this test to detect protocol level blocks.
We can maybe do this incrementally and iteratively, by first having a simple prototype that just a basic reachability test and then expanding it in the future to also generate traffic.
With https://github.com/ooni/probe-cli/pull/891, we merged an initial prototype to start building from. 🥳