ooni
Add tcp_connect support to the TCPTest template
Currently in many tests we have a pattern for performing a TCP connect test. In particular we do this in:
-
Web Connectivity
-
Facebook Messenger
-
WhatsApp
It's worthwhile pointing out that there is currently also some inconsistency between the format of the tcp_connect key between web_connectivity and facebook_messenger and whatsapp.
In particular web_connectivity uses a dictionary keyed on the ip:port combination, while facebook_messenger and whatsapp use a list.
I believe using a list is more robust as we actually care to log when multiple connects fail to the same endpoint.
Ideally this would be supported by the TCPTest template so that we ensure the format is always consistent and it's easy to redo without too much boilerplate.
A nice addition would be if the sendPayload method also used this method before starting to send data.
The data format should look something like the following:
{
"tcp_connect": [
{
"ip": "xxx.xxx.xxx.xxx",
"port": int
"status": {
"success": true | false,
"failure": "FAILURE STRING",
"start_time": "TIMESTAMP",
"rtt":"double indicating the time it took to get the failure or successfully connect"
}
}
]
}
I believe we could then rename "sent" and "received" to something like "tcp_send" and "tcp_recv"
{
"sent": [
"ASCII",
{"format": "bas64", "data": "BASE64 ENCODED BINARY DATA"},
"ASCII"
],
"received": {"format": "bas64", "data": "BASE64 ENCODED BINARY DATA"} | "str",
"stats": [
{
"sent_time": TIMESTAMP,
"first_payload": "double indicating how many milliseconds it took to receive the first payload after the send",
"last_payload": "double indicating how many milliseconds it took to receive the last payload after the send"
}
]
}
These keys should be nested under the tcp_connect array that refers to the session this is relevant to like so:
{
"tcp_connect": [
{
"ip": "xxx.xxx.xxx.xxx",
"port": int
"status": {
"success": true | false,
"failure": "FAILURE STRING",
"start_time": "TIMESTAMP",
"rtt":"double indicating the time it took to get the failure or successfully connect"
},
"sent": [],
"received": "",
"stats": []
}
]
}
The stats list is indexed on the sent array.
I am a bit unsure about the format there and if the format for that is sane.
"rtt": "double indicating the time it took to get the failure or successfully connect"
(That's a reminder to future self)
Time-to-connect not RTT in case of retransmissions, so I'd rather name it differently. Firefox developer tools name that Connecting, Chrome names that Initial connection.
tcp_info may suggest some RTT value based on TCP timestamps, IIRC, but it should be polled to get data as ACKs do not trigger any activity in userspace.
