XEP-0138: Experimental implementation of Stream Compression

Open horazont opened this issue 7 years ago • 2 comments

Despite being unsafe with a naive implementation, I’d like to have one to be able to play around with it. I would like to benchmark the different strategies to make Stream Compression safe (flushing after each stanza, flushing after each different (from,to) pair, etc.).

The support MUST NOT be used except if explicitly enabled via a hidden switch, because the naive implementation is vulnerable.

Nov 01 '18 13:11 horazont

So, I have an experimental implementation locally. I used it to benchmark compression in various situations. Putting the data here for future use:

The following are JSON arrays with recordings of the connections in the respective scenarios. tx_pre_filter is number of bytes sent by the client, before compression. tx_post_filter is number of bytes sent after compression. rx_pre_filter is number of bytes received before decompression and rx_post_filter is number of bytes received after decompression.

aioxmpp-e2etest-full_flush

[
{"tx_pre_filter": 1480, "tx_post_filter": 1460, "rx_pre_filter": 3587, "rx_post_filter": 7971},
{"tx_pre_filter": 1227, "tx_post_filter": 1145, "rx_pre_filter": 2395, "rx_post_filter": 2700},
{"tx_pre_filter": 1191, "tx_post_filter": 1131, "rx_pre_filter": 2471, "rx_post_filter": 2791},
{"tx_pre_filter": 1191, "tx_post_filter": 1130, "rx_pre_filter": 2480, "rx_post_filter": 2791},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2024, "rx_post_filter": 2175},
{"tx_pre_filter": 1432, "tx_post_filter": 1245, "rx_pre_filter": 2397, "rx_post_filter": 2747},
{"tx_pre_filter": 1699, "tx_post_filter": 1505, "rx_pre_filter": 2662, "rx_post_filter": 3057},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2024, "rx_post_filter": 2175},
{"tx_pre_filter": 1364, "tx_post_filter": 1226, "rx_pre_filter": 2378, "rx_post_filter": 2689},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2026, "rx_post_filter": 2175},
{"tx_pre_filter": 1574, "tx_post_filter": 1279, "rx_pre_filter": 2428, "rx_post_filter": 2889},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2023, "rx_post_filter": 2175},
{"tx_pre_filter": 1370, "tx_post_filter": 1213, "rx_pre_filter": 2363, "rx_post_filter": 2685},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2027, "rx_post_filter": 2175},
{"tx_pre_filter": 8134, "tx_post_filter": 6170, "rx_pre_filter": 3393, "rx_post_filter": 5842},
{"tx_pre_filter": 8368, "tx_post_filter": 6374, "rx_pre_filter": 3558, "rx_post_filter": 6028},
{"tx_pre_filter": 2415, "tx_post_filter": 2186, "rx_pre_filter": 3271, "rx_post_filter": 3531},
{"tx_pre_filter": 2415, "tx_post_filter": 2195, "rx_pre_filter": 3282, "rx_post_filter": 3531},
{"tx_pre_filter": 2415, "tx_post_filter": 2195, "rx_pre_filter": 3293, "rx_post_filter": 3531},
{"tx_pre_filter": 2415, "tx_post_filter": 2187, "rx_pre_filter": 3279, "rx_post_filter": 3531},
{"tx_pre_filter": 2415, "tx_post_filter": 2192, "rx_pre_filter": 3277, "rx_post_filter": 3531},
{"tx_pre_filter": 2415, "tx_post_filter": 2193, "rx_pre_filter": 3285, "rx_post_filter": 3531},
{"tx_pre_filter": 1370, "tx_post_filter": 1170, "rx_pre_filter": 2298, "rx_post_filter": 2642},
{"tx_pre_filter": 1370, "tx_post_filter": 1171, "rx_pre_filter": 2303, "rx_post_filter": 2642},
{"tx_pre_filter": 1560, "tx_post_filter": 1340, "rx_pre_filter": 2722, "rx_post_filter": 3176},
{"tx_pre_filter": 1394, "tx_post_filter": 1269, "rx_pre_filter": 2229, "rx_post_filter": 2408},
{"tx_pre_filter": 1370, "tx_post_filter": 1171, "rx_pre_filter": 2300, "rx_post_filter": 2642},
{"tx_pre_filter": 1370, "tx_post_filter": 1171, "rx_pre_filter": 2304, "rx_post_filter": 2642},
{"tx_pre_filter": 1410, "tx_post_filter": 1277, "rx_pre_filter": 2257, "rx_post_filter": 2434},
{"tx_pre_filter": 1586, "tx_post_filter": 1372, "rx_pre_filter": 2837, "rx_post_filter": 3306},
{"tx_pre_filter": 2231, "tx_post_filter": 1960, "rx_pre_filter": 3915, "rx_post_filter": 5176},
{"tx_pre_filter": 3861, "tx_post_filter": 2745, "rx_pre_filter": 3208, "rx_post_filter": 3675},
{"tx_pre_filter": 1125, "tx_post_filter": 1104, "rx_pre_filter": 2479, "rx_post_filter": 3133},
{"tx_pre_filter": 1019, "tx_post_filter": 1001, "rx_pre_filter": 2338, "rx_post_filter": 2984},
{"tx_pre_filter": 1125, "tx_post_filter": 1104, "rx_pre_filter": 2471, "rx_post_filter": 3133},
{"tx_pre_filter": 1019, "tx_post_filter": 1001, "rx_pre_filter": 2339, "rx_post_filter": 2984},
{"tx_pre_filter": 1019, "tx_post_filter": 1001, "rx_pre_filter": 2338, "rx_post_filter": 2984},
{"tx_pre_filter": 1019, "tx_post_filter": 1001, "rx_pre_filter": 2341, "rx_post_filter": 2984},
{"tx_pre_filter": 1105, "tx_post_filter": 1057, "rx_pre_filter": 2195, "rx_post_filter": 2372},
{"tx_pre_filter": 1100, "tx_post_filter": 1056, "rx_pre_filter": 2229, "rx_post_filter": 2420},
{"tx_pre_filter": 1019, "tx_post_filter": 1000, "rx_pre_filter": 2338, "rx_post_filter": 2984},
{"tx_pre_filter": 1147, "tx_post_filter": 1118, "rx_pre_filter": 3064, "rx_post_filter": 3957},
{"tx_pre_filter": 4637, "tx_post_filter": 2435, "rx_pre_filter": 4556, "rx_post_filter": 7564},
{"tx_pre_filter": 1147, "tx_post_filter": 1121, "rx_pre_filter": 3084, "rx_post_filter": 3971},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2337, "rx_post_filter": 2984},
{"tx_pre_filter": 4651, "tx_post_filter": 2451, "rx_pre_filter": 4572, "rx_post_filter": 7578},
{"tx_pre_filter": 1019, "tx_post_filter": 997, "rx_pre_filter": 2510, "rx_post_filter": 3193},
{"tx_pre_filter": 1147, "tx_post_filter": 1119, "rx_pre_filter": 2843, "rx_post_filter": 3606},
{"tx_pre_filter": 4578, "tx_post_filter": 2403, "rx_pre_filter": 4395, "rx_post_filter": 7328},
{"tx_pre_filter": 1147, "tx_post_filter": 1120, "rx_pre_filter": 3036, "rx_post_filter": 3829},
{"tx_pre_filter": 4412, "tx_post_filter": 2264, "rx_pre_filter": 4607, "rx_post_filter": 7593},
{"tx_pre_filter": 1146, "tx_post_filter": 1122, "rx_pre_filter": 3350, "rx_post_filter": 4247},
{"tx_pre_filter": 1172, "tx_post_filter": 1137, "rx_pre_filter": 3283, "rx_post_filter": 4191},
{"tx_pre_filter": 4565, "tx_post_filter": 2399, "rx_pre_filter": 4802, "rx_post_filter": 7799},
{"tx_pre_filter": 1147, "tx_post_filter": 1123, "rx_pre_filter": 3554, "rx_post_filter": 4453},
{"tx_pre_filter": 1019, "tx_post_filter": 998, "rx_pre_filter": 2333, "rx_post_filter": 2984},
{"tx_pre_filter": 4627, "tx_post_filter": 2431, "rx_pre_filter": 4554, "rx_post_filter": 7575},
{"tx_pre_filter": 1147, "tx_post_filter": 1116, "rx_pre_filter": 3057, "rx_post_filter": 3968},
{"tx_pre_filter": 1147, "tx_post_filter": 1121, "rx_pre_filter": 2839, "rx_post_filter": 3606},
{"tx_pre_filter": 4412, "tx_post_filter": 2263, "rx_pre_filter": 4399, "rx_post_filter": 7328},
{"tx_pre_filter": 1019, "tx_post_filter": 1001, "rx_pre_filter": 2337, "rx_post_filter": 2984},
{"tx_pre_filter": 4627, "tx_post_filter": 2431, "rx_pre_filter": 4550, "rx_post_filter": 7575},
{"tx_pre_filter": 1147, "tx_post_filter": 1122, "rx_pre_filter": 3070, "rx_post_filter": 3968},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2338, "rx_post_filter": 2984},
{"tx_pre_filter": 1019, "tx_post_filter": 998, "rx_pre_filter": 2332, "rx_post_filter": 2984},
{"tx_pre_filter": 4412, "tx_post_filter": 2262, "rx_pre_filter": 4421, "rx_post_filter": 7355},
{"tx_pre_filter": 1229, "tx_post_filter": 1204, "rx_pre_filter": 3059, "rx_post_filter": 3904},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2581, "rx_post_filter": 3432},
{"tx_pre_filter": 1147, "tx_post_filter": 1125, "rx_pre_filter": 2839, "rx_post_filter": 3606},
{"tx_pre_filter": 4629, "tx_post_filter": 2432, "rx_pre_filter": 4388, "rx_post_filter": 7328},
{"tx_pre_filter": 4412, "tx_post_filter": 2260, "rx_pre_filter": 4392, "rx_post_filter": 7303},
{"tx_pre_filter": 1019, "tx_post_filter": 1001, "rx_pre_filter": 2338, "rx_post_filter": 2984},
{"tx_pre_filter": 1276, "tx_post_filter": 1247, "rx_pre_filter": 3021, "rx_post_filter": 3815},
{"tx_pre_filter": 4564, "tx_post_filter": 2395, "rx_pre_filter": 4358, "rx_post_filter": 7283},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2335, "rx_post_filter": 2984},
{"tx_pre_filter": 1147, "tx_post_filter": 1120, "rx_pre_filter": 3178, "rx_post_filter": 4021},
{"tx_pre_filter": 1019, "tx_post_filter": 1001, "rx_pre_filter": 2338, "rx_post_filter": 2984},
{"tx_pre_filter": 4564, "tx_post_filter": 2392, "rx_pre_filter": 4376, "rx_post_filter": 7283},
{"tx_pre_filter": 1147, "tx_post_filter": 1125, "rx_pre_filter": 3207, "rx_post_filter": 4021},
{"tx_pre_filter": 1019, "tx_post_filter": 997, "rx_pre_filter": 2336, "rx_post_filter": 2984},
{"tx_pre_filter": 1147, "tx_post_filter": 1123, "rx_pre_filter": 3220, "rx_post_filter": 4072},
{"tx_pre_filter": 4482, "tx_post_filter": 2340, "rx_pre_filter": 4820, "rx_post_filter": 7918},
{"tx_pre_filter": 1147, "tx_post_filter": 1123, "rx_pre_filter": 2942, "rx_post_filter": 3753},
{"tx_pre_filter": 4532, "tx_post_filter": 2368, "rx_pre_filter": 4504, "rx_post_filter": 7475},
{"tx_pre_filter": 1019, "tx_post_filter": 998, "rx_pre_filter": 2334, "rx_post_filter": 2984},
{"tx_pre_filter": 1434, "tx_post_filter": 1326, "rx_pre_filter": 3215, "rx_post_filter": 4095},
{"tx_pre_filter": 1019, "tx_post_filter": 998, "rx_pre_filter": 2333, "rx_post_filter": 2984},
{"tx_pre_filter": 7976, "tx_post_filter": 3679, "rx_pre_filter": 6556, "rx_post_filter": 12497},
{"tx_pre_filter": 2268, "tx_post_filter": 1813, "rx_pre_filter": 3544, "rx_post_filter": 6053},
{"tx_pre_filter": 2665, "tx_post_filter": 1973, "rx_pre_filter": 3557, "rx_post_filter": 7629},
{"tx_pre_filter": 2666, "tx_post_filter": 1979, "rx_pre_filter": 3572, "rx_post_filter": 7628},
{"tx_pre_filter": 2318, "tx_post_filter": 1846, "rx_pre_filter": 4084, "rx_post_filter": 9826},
{"tx_pre_filter": 1763, "tx_post_filter": 1478, "rx_pre_filter": 3167, "rx_post_filter": 5505},
{"tx_pre_filter": 2330, "tx_post_filter": 2044, "rx_pre_filter": 4503, "rx_post_filter": 7037},
{"tx_pre_filter": 1901, "tx_post_filter": 1718, "rx_pre_filter": 4221, "rx_post_filter": 4942},
{"tx_pre_filter": 2330, "tx_post_filter": 2041, "rx_pre_filter": 4490, "rx_post_filter": 7037},
{"tx_pre_filter": 1901, "tx_post_filter": 1718, "rx_pre_filter": 4211, "rx_post_filter": 4942},
{"tx_pre_filter": 2056, "tx_post_filter": 1737, "rx_pre_filter": 3528, "rx_post_filter": 5911},
{"tx_pre_filter": 1627, "tx_post_filter": 1413, "rx_pre_filter": 2624, "rx_post_filter": 2994},
{"tx_pre_filter": 2330, "tx_post_filter": 2041, "rx_pre_filter": 4478, "rx_post_filter": 7021},
{"tx_pre_filter": 1901, "tx_post_filter": 1715, "rx_pre_filter": 4224, "rx_post_filter": 4942},
{"tx_pre_filter": 1020, "tx_post_filter": 1003, "rx_pre_filter": 2217, "rx_post_filter": 2403},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2024, "rx_post_filter": 2175},
{"tx_pre_filter": 1131, "tx_post_filter": 1073, "rx_pre_filter": 2168, "rx_post_filter": 2335},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2026, "rx_post_filter": 2175},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2023, "rx_post_filter": 2175},
{"tx_pre_filter": 994, "tx_post_filter": 980, "rx_pre_filter": 2129, "rx_post_filter": 2279},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2022, "rx_post_filter": 2175},
{"tx_pre_filter": 1045, "tx_post_filter": 1023, "rx_pre_filter": 2274, "rx_post_filter": 2540},
{"tx_pre_filter": 1268, "tx_post_filter": 1133, "rx_pre_filter": 2194, "rx_post_filter": 2360},
{"tx_pre_filter": 1020, "tx_post_filter": 1001, "rx_pre_filter": 2175, "rx_post_filter": 2338},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2022, "rx_post_filter": 2175},
{"tx_pre_filter": 1066, "tx_post_filter": 1035, "rx_pre_filter": 2171, "rx_post_filter": 2335},
{"tx_pre_filter": 1020, "tx_post_filter": 1005, "rx_pre_filter": 2170, "rx_post_filter": 2338},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2023, "rx_post_filter": 2175},
{"tx_pre_filter": 1066, "tx_post_filter": 1033, "rx_pre_filter": 2170, "rx_post_filter": 2335},
{"tx_pre_filter": 1069, "tx_post_filter": 1054, "rx_pre_filter": 2022, "rx_post_filter": 2175},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2369, "rx_post_filter": 2532},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2135, "rx_post_filter": 2288},
{"tx_pre_filter": 973, "tx_post_filter": 961, "rx_pre_filter": 2023, "rx_post_filter": 2175},
{"tx_pre_filter": 1185, "tx_post_filter": 1132, "rx_pre_filter": 2269, "rx_post_filter": 2435},
{"tx_pre_filter": 1199, "tx_post_filter": 1136, "rx_pre_filter": 2266, "rx_post_filter": 2437},
{"tx_pre_filter": 1084, "tx_post_filter": 1041, "rx_pre_filter": 2188, "rx_post_filter": 2372},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2023, "rx_post_filter": 2175},
{"tx_pre_filter": 1257, "tx_post_filter": 1275, "rx_pre_filter": 3188, "rx_post_filter": 3486},
{"tx_pre_filter": 1257, "tx_post_filter": 1274, "rx_pre_filter": 3138, "rx_post_filter": 3409},
{"tx_pre_filter": 1257, "tx_post_filter": 1275, "rx_pre_filter": 3040, "rx_post_filter": 3304},
{"tx_pre_filter": 1257, "tx_post_filter": 1271, "rx_pre_filter": 3296, "rx_post_filter": 3591},
{"tx_pre_filter": 1136, "tx_post_filter": 1142, "rx_pre_filter": 2784, "rx_post_filter": 3023},
{"tx_pre_filter": 1104, "tx_post_filter": 1104, "rx_pre_filter": 2403, "rx_post_filter": 2570},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2023, "rx_post_filter": 2175},
{"tx_pre_filter": 999, "tx_post_filter": 987, "rx_pre_filter": 2199, "rx_post_filter": 2392},
{"tx_pre_filter": 1025, "tx_post_filter": 1005, "rx_pre_filter": 2218, "rx_post_filter": 2403},
{"tx_pre_filter": 1131, "tx_post_filter": 1073, "rx_pre_filter": 2168, "rx_post_filter": 2340},
{"tx_pre_filter": 1025, "tx_post_filter": 1010, "rx_pre_filter": 2230, "rx_post_filter": 2423},
{"tx_pre_filter": 1151, "tx_post_filter": 1093, "rx_pre_filter": 2179, "rx_post_filter": 2340},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2025, "rx_post_filter": 2175},
{"tx_pre_filter": 1041, "tx_post_filter": 1015, "rx_pre_filter": 2019, "rx_post_filter": 2175},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2185, "rx_post_filter": 2356},
{"tx_pre_filter": 1006, "tx_post_filter": 986, "rx_pre_filter": 2133, "rx_post_filter": 2321},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2082, "rx_post_filter": 2078},
{"tx_pre_filter": 1245, "tx_post_filter": 1164, "rx_pre_filter": 2440, "rx_post_filter": 2316},
{"tx_pre_filter": 1008, "tx_post_filter": 991, "rx_pre_filter": 2184, "rx_post_filter": 2360},
{"tx_pre_filter": 976, "tx_post_filter": 959, "rx_pre_filter": 2178, "rx_post_filter": 2360}
]

aioxmpp-e2etest-sync_flush

[
{"tx_pre_filter": 1480, "tx_post_filter": 1455, "rx_pre_filter": 2694, "rx_post_filter": 7971},
{"tx_pre_filter": 1227, "tx_post_filter": 1144, "rx_pre_filter": 2060, "rx_post_filter": 2700},
{"tx_pre_filter": 1191, "tx_post_filter": 1128, "rx_pre_filter": 2167, "rx_post_filter": 2791},
{"tx_pre_filter": 1191, "tx_post_filter": 1134, "rx_pre_filter": 2168, "rx_post_filter": 2791},
{"tx_pre_filter": 1432, "tx_post_filter": 1243, "rx_pre_filter": 2063, "rx_post_filter": 2747},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1898, "rx_post_filter": 2175},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1898, "rx_post_filter": 2175},
{"tx_pre_filter": 1699, "tx_post_filter": 1531, "rx_pre_filter": 2067, "rx_post_filter": 3057},
{"tx_pre_filter": 1364, "tx_post_filter": 1223, "rx_pre_filter": 2063, "rx_post_filter": 2689},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1900, "rx_post_filter": 2175},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1899, "rx_post_filter": 2175},
{"tx_pre_filter": 1574, "tx_post_filter": 1284, "rx_pre_filter": 2098, "rx_post_filter": 2889},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1899, "rx_post_filter": 2175},
{"tx_pre_filter": 1370, "tx_post_filter": 1217, "rx_pre_filter": 2031, "rx_post_filter": 2685},
{"tx_pre_filter": 8677, "tx_post_filter": 6453, "rx_pre_filter": 2826, "rx_post_filter": 6068},
{"tx_pre_filter": 8368, "tx_post_filter": 6373, "rx_pre_filter": 2810, "rx_post_filter": 6028},
{"tx_pre_filter": 2415, "tx_post_filter": 2194, "rx_pre_filter": 2266, "rx_post_filter": 3531},
{"tx_pre_filter": 2415, "tx_post_filter": 2197, "rx_pre_filter": 2263, "rx_post_filter": 3531},
{"tx_pre_filter": 2415, "tx_post_filter": 2197, "rx_pre_filter": 2264, "rx_post_filter": 3531},
{"tx_pre_filter": 2415, "tx_post_filter": 2194, "rx_pre_filter": 2260, "rx_post_filter": 3531},
{"tx_pre_filter": 2415, "tx_post_filter": 2185, "rx_pre_filter": 2263, "rx_post_filter": 3531},
{"tx_pre_filter": 2415, "tx_post_filter": 2194, "rx_pre_filter": 2261, "rx_post_filter": 3531},
{"tx_pre_filter": 1824, "tx_post_filter": 1434, "rx_pre_filter": 2204, "rx_post_filter": 2868},
{"tx_pre_filter": 1824, "tx_post_filter": 1432, "rx_pre_filter": 2207, "rx_post_filter": 2868},
{"tx_pre_filter": 1394, "tx_post_filter": 1264, "rx_pre_filter": 2025, "rx_post_filter": 2408},
{"tx_pre_filter": 1560, "tx_post_filter": 1339, "rx_pre_filter": 2353, "rx_post_filter": 3290},
{"tx_pre_filter": 1370, "tx_post_filter": 1170, "rx_pre_filter": 2132, "rx_post_filter": 2642},
{"tx_pre_filter": 1370, "tx_post_filter": 1170, "rx_pre_filter": 2130, "rx_post_filter": 2642},
{"tx_pre_filter": 1586, "tx_post_filter": 1370, "rx_pre_filter": 2341, "rx_post_filter": 3192},
{"tx_pre_filter": 1410, "tx_post_filter": 1272, "rx_pre_filter": 2050, "rx_post_filter": 2434},
{"tx_pre_filter": 2231, "tx_post_filter": 1963, "rx_pre_filter": 2474, "rx_post_filter": 5176},
{"tx_pre_filter": 3861, "tx_post_filter": 2772, "rx_pre_filter": 2358, "rx_post_filter": 3675},
{"tx_pre_filter": 1125, "tx_post_filter": 1106, "rx_pre_filter": 2218, "rx_post_filter": 3133},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2127, "rx_post_filter": 2984},
{"tx_pre_filter": 1125, "tx_post_filter": 1104, "rx_pre_filter": 2216, "rx_post_filter": 3133},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2129, "rx_post_filter": 2984},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2127, "rx_post_filter": 2984},
{"tx_pre_filter": 1019, "tx_post_filter": 1000, "rx_pre_filter": 2127, "rx_post_filter": 2984},
{"tx_pre_filter": 1105, "tx_post_filter": 1056, "rx_pre_filter": 2019, "rx_post_filter": 2372},
{"tx_pre_filter": 1100, "tx_post_filter": 1055, "rx_pre_filter": 2041, "rx_post_filter": 2420},
{"tx_pre_filter": 1147, "tx_post_filter": 1123, "rx_pre_filter": 2451, "rx_post_filter": 3957},
{"tx_pre_filter": 1019, "tx_post_filter": 1000, "rx_pre_filter": 2126, "rx_post_filter": 2984},
{"tx_pre_filter": 4637, "tx_post_filter": 2436, "rx_pre_filter": 3512, "rx_post_filter": 7564},
{"tx_pre_filter": 1147, "tx_post_filter": 1122, "rx_pre_filter": 2463, "rx_post_filter": 3971},
{"tx_pre_filter": 4651, "tx_post_filter": 2454, "rx_pre_filter": 3527, "rx_post_filter": 7578},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2128, "rx_post_filter": 2984},
{"tx_pre_filter": 1147, "tx_post_filter": 1122, "rx_pre_filter": 2364, "rx_post_filter": 3606},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2260, "rx_post_filter": 3193},
{"tx_pre_filter": 4578, "tx_post_filter": 2410, "rx_pre_filter": 3424, "rx_post_filter": 7328},
{"tx_pre_filter": 1147, "tx_post_filter": 1119, "rx_pre_filter": 2405, "rx_post_filter": 3829},
{"tx_pre_filter": 1146, "tx_post_filter": 1120, "rx_pre_filter": 2434, "rx_post_filter": 4038},
{"tx_pre_filter": 4412, "tx_post_filter": 2261, "rx_pre_filter": 3505, "rx_post_filter": 7843},
{"tx_pre_filter": 1172, "tx_post_filter": 1136, "rx_pre_filter": 2515, "rx_post_filter": 4191},
{"tx_pre_filter": 4565, "tx_post_filter": 2396, "rx_pre_filter": 3545, "rx_post_filter": 7799},
{"tx_pre_filter": 1147, "tx_post_filter": 1117, "rx_pre_filter": 2505, "rx_post_filter": 4453},
{"tx_pre_filter": 1147, "tx_post_filter": 1118, "rx_pre_filter": 2453, "rx_post_filter": 3968},
{"tx_pre_filter": 4627, "tx_post_filter": 2438, "rx_pre_filter": 3521, "rx_post_filter": 7575},
{"tx_pre_filter": 1019, "tx_post_filter": 998, "rx_pre_filter": 2125, "rx_post_filter": 2984},
{"tx_pre_filter": 1147, "tx_post_filter": 1112, "rx_pre_filter": 2359, "rx_post_filter": 3606},
{"tx_pre_filter": 1019, "tx_post_filter": 998, "rx_pre_filter": 2127, "rx_post_filter": 2984},
{"tx_pre_filter": 4412, "tx_post_filter": 2260, "rx_pre_filter": 3425, "rx_post_filter": 7328},
{"tx_pre_filter": 1147, "tx_post_filter": 1122, "rx_pre_filter": 2454, "rx_post_filter": 3968},
{"tx_pre_filter": 4627, "tx_post_filter": 2436, "rx_pre_filter": 3524, "rx_post_filter": 7575},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2129, "rx_post_filter": 2984},
{"tx_pre_filter": 1229, "tx_post_filter": 1211, "rx_pre_filter": 2415, "rx_post_filter": 3904},
{"tx_pre_filter": 4412, "tx_post_filter": 2256, "rx_pre_filter": 3444, "rx_post_filter": 7355},
{"tx_pre_filter": 1019, "tx_post_filter": 1001, "rx_pre_filter": 2129, "rx_post_filter": 2984},
{"tx_pre_filter": 1147, "tx_post_filter": 1121, "rx_pre_filter": 2364, "rx_post_filter": 3606},
{"tx_pre_filter": 1019, "tx_post_filter": 1003, "rx_pre_filter": 2316, "rx_post_filter": 3432},
{"tx_pre_filter": 4629, "tx_post_filter": 2432, "rx_pre_filter": 3417, "rx_post_filter": 7328},
{"tx_pre_filter": 1019, "tx_post_filter": 1000, "rx_pre_filter": 2125, "rx_post_filter": 2984},
{"tx_pre_filter": 4412, "tx_post_filter": 2266, "rx_pre_filter": 3469, "rx_post_filter": 7303},
{"tx_pre_filter": 1276, "tx_post_filter": 1247, "rx_pre_filter": 2395, "rx_post_filter": 3815},
{"tx_pre_filter": 1147, "tx_post_filter": 1116, "rx_pre_filter": 2417, "rx_post_filter": 3812},
{"tx_pre_filter": 1019, "tx_post_filter": 997, "rx_pre_filter": 2125, "rx_post_filter": 2984},
{"tx_pre_filter": 4564, "tx_post_filter": 2398, "rx_pre_filter": 3475, "rx_post_filter": 7534},
{"tx_pre_filter": 4564, "tx_post_filter": 2401, "rx_pre_filter": 3454, "rx_post_filter": 7283},
{"tx_pre_filter": 1147, "tx_post_filter": 1119, "rx_pre_filter": 2446, "rx_post_filter": 4021},
{"tx_pre_filter": 1019, "tx_post_filter": 1001, "rx_pre_filter": 2127, "rx_post_filter": 2984},
{"tx_pre_filter": 1019, "tx_post_filter": 997, "rx_pre_filter": 2126, "rx_post_filter": 2984},
{"tx_pre_filter": 1147, "tx_post_filter": 1122, "rx_pre_filter": 2450, "rx_post_filter": 4072},
{"tx_pre_filter": 4482, "tx_post_filter": 2341, "rx_pre_filter": 3495, "rx_post_filter": 7918},
{"tx_pre_filter": 4532, "tx_post_filter": 2375, "rx_pre_filter": 3433, "rx_post_filter": 7224},
{"tx_pre_filter": 1019, "tx_post_filter": 1000, "rx_pre_filter": 2126, "rx_post_filter": 2984},
{"tx_pre_filter": 1147, "tx_post_filter": 1124, "rx_pre_filter": 2427, "rx_post_filter": 3962},
{"tx_pre_filter": 1434, "tx_post_filter": 1325, "rx_pre_filter": 2423, "rx_post_filter": 4095},
{"tx_pre_filter": 7976, "tx_post_filter": 3681, "rx_pre_filter": 3828, "rx_post_filter": 12497},
{"tx_pre_filter": 1019, "tx_post_filter": 999, "rx_pre_filter": 2127, "rx_post_filter": 2984},
{"tx_pre_filter": 2801, "tx_post_filter": 2090, "rx_pre_filter": 2973, "rx_post_filter": 6279},
{"tx_pre_filter": 2665, "tx_post_filter": 1977, "rx_pre_filter": 2756, "rx_post_filter": 7629},
{"tx_pre_filter": 2666, "tx_post_filter": 1974, "rx_pre_filter": 2727, "rx_post_filter": 7628},
{"tx_pre_filter": 2318, "tx_post_filter": 1852, "rx_pre_filter": 3255, "rx_post_filter": 9826},
{"tx_pre_filter": 1763, "tx_post_filter": 1482, "rx_pre_filter": 2731, "rx_post_filter": 5505},
{"tx_pre_filter": 2869, "tx_post_filter": 2330, "rx_pre_filter": 3201, "rx_post_filter": 7263},
{"tx_pre_filter": 2440, "tx_post_filter": 2002, "rx_pre_filter": 2855, "rx_post_filter": 5168},
{"tx_pre_filter": 2330, "tx_post_filter": 2035, "rx_pre_filter": 3157, "rx_post_filter": 7037},
{"tx_pre_filter": 1901, "tx_post_filter": 1715, "rx_pre_filter": 2806, "rx_post_filter": 4942},
{"tx_pre_filter": 2056, "tx_post_filter": 1735, "rx_pre_filter": 2843, "rx_post_filter": 5911},
{"tx_pre_filter": 1627, "tx_post_filter": 1411, "rx_pre_filter": 2256, "rx_post_filter": 2994},
{"tx_pre_filter": 2330, "tx_post_filter": 2038, "rx_pre_filter": 3155, "rx_post_filter": 7021},
{"tx_pre_filter": 1901, "tx_post_filter": 1714, "rx_pre_filter": 2810, "rx_post_filter": 4942},
{"tx_pre_filter": 1020, "tx_post_filter": 1006, "rx_pre_filter": 2015, "rx_post_filter": 2403},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1899, "rx_post_filter": 2175},
{"tx_pre_filter": 1131, "tx_post_filter": 1076, "rx_pre_filter": 1981, "rx_post_filter": 2335},
{"tx_pre_filter": 994, "tx_post_filter": 980, "rx_pre_filter": 1940, "rx_post_filter": 2279},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1897, "rx_post_filter": 2175},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1897, "rx_post_filter": 2175},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1899, "rx_post_filter": 2175},
{"tx_pre_filter": 1045, "tx_post_filter": 1021, "rx_pre_filter": 2064, "rx_post_filter": 2540},
{"tx_pre_filter": 1268, "tx_post_filter": 1134, "rx_pre_filter": 1993, "rx_post_filter": 2360},
{"tx_pre_filter": 1020, "tx_post_filter": 1005, "rx_pre_filter": 1980, "rx_post_filter": 2338},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1900, "rx_post_filter": 2175},
{"tx_pre_filter": 1066, "tx_post_filter": 1037, "rx_pre_filter": 1982, "rx_post_filter": 2335},
{"tx_pre_filter": 1020, "tx_post_filter": 1005, "rx_pre_filter": 1979, "rx_post_filter": 2338},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1899, "rx_post_filter": 2175},
{"tx_pre_filter": 1066, "tx_post_filter": 1035, "rx_pre_filter": 1982, "rx_post_filter": 2335},
{"tx_pre_filter": 1069, "tx_post_filter": 1054, "rx_pre_filter": 1897, "rx_post_filter": 2175},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2015, "rx_post_filter": 2532},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1962, "rx_post_filter": 2288},
{"tx_pre_filter": 973, "tx_post_filter": 959, "rx_pre_filter": 1899, "rx_post_filter": 2175},
{"tx_pre_filter": 1185, "tx_post_filter": 1130, "rx_pre_filter": 2014, "rx_post_filter": 2435},
{"tx_pre_filter": 1199, "tx_post_filter": 1131, "rx_pre_filter": 2017, "rx_post_filter": 2437},
{"tx_pre_filter": 1084, "tx_post_filter": 1043, "rx_pre_filter": 1986, "rx_post_filter": 2372},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1900, "rx_post_filter": 2175},
{"tx_pre_filter": 1257, "tx_post_filter": 1274, "rx_pre_filter": 2296, "rx_post_filter": 3486},
{"tx_pre_filter": 1257, "tx_post_filter": 1271, "rx_pre_filter": 2304, "rx_post_filter": 3409},
{"tx_pre_filter": 1257, "tx_post_filter": 1275, "rx_pre_filter": 2294, "rx_post_filter": 3486},
{"tx_pre_filter": 1257, "tx_post_filter": 1273, "rx_pre_filter": 2304, "rx_post_filter": 3409},
{"tx_pre_filter": 1136, "tx_post_filter": 1141, "rx_pre_filter": 2226, "rx_post_filter": 3023},
{"tx_pre_filter": 1104, "tx_post_filter": 1104, "rx_pre_filter": 2078, "rx_post_filter": 2570},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1899, "rx_post_filter": 2175},
{"tx_pre_filter": 999, "tx_post_filter": 982, "rx_pre_filter": 2010, "rx_post_filter": 2392},
{"tx_pre_filter": 1025, "tx_post_filter": 1009, "rx_pre_filter": 2016, "rx_post_filter": 2403},
{"tx_pre_filter": 1131, "tx_post_filter": 1073, "rx_pre_filter": 1986, "rx_post_filter": 2340},
{"tx_pre_filter": 1025, "tx_post_filter": 1008, "rx_pre_filter": 2043, "rx_post_filter": 2423},
{"tx_pre_filter": 1151, "tx_post_filter": 1090, "rx_pre_filter": 1987, "rx_post_filter": 2340},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1898, "rx_post_filter": 2175},
{"tx_pre_filter": 1041, "tx_post_filter": 1017, "rx_pre_filter": 1898, "rx_post_filter": 2175},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 2006, "rx_post_filter": 2356},
{"tx_pre_filter": 1006, "tx_post_filter": 986, "rx_pre_filter": 1961, "rx_post_filter": 2321},
{"tx_pre_filter": 903, "tx_post_filter": 885, "rx_pre_filter": 1955, "rx_post_filter": 2078},
{"tx_pre_filter": 1245, "tx_post_filter": 1161, "rx_pre_filter": 2148, "rx_post_filter": 2316},
{"tx_pre_filter": 1008, "tx_post_filter": 989, "rx_pre_filter": 1986, "rx_post_filter": 2360},
{"tx_pre_filter": 976, "tx_post_filter": 959, "rx_pre_filter": 1986, "rx_post_filter": 2360}
]

`jabbercat-login-full_flush`

[
{"tx_pre_filter": 67944, "tx_post_filter": 60112, "rx_pre_filter": 1800022, "rx_post_filter": 2440004},
{"tx_pre_filter": 10646, "tx_post_filter": 9144, "rx_pre_filter": 121255, "rx_post_filter": 178321}
]

`jabbercat-login-sync_flush`

[
{"tx_pre_filter": 10968, "tx_post_filter": 9449, "rx_pre_filter": 85906, "rx_post_filter": 181490},
{"tx_pre_filter": 85376, "tx_post_filter": 76158, "rx_pre_filter": 1745398, "rx_post_filter": 2685454}
]

This results in the following ratios (the percentage being the ratio of bytes saved; tx is from client to server, rx is from server to client):

aioxmpp test suite, sync_flush only (XEP-0138 as written): 40% rx, 20% tx
aioxmpp test suite, full_flush after each stanza: 25% rx, 20% tx
JabberCat startup (lots of mucs, lots of avatars), full flush after each stanza: 25% rx, 12% tx
JabberCat startup (lots of mucs, lots of avatars), sync flush: 36% rx, 12% tx

Nov 02 '18 16:11 horazont

We have a user who’d like to have XEP-0138. Maybe we can get an implementation together which is fully opt-in?

Jan 15 '19 15:01 horazont