Suggestion: Fast HTTP reconnections

[lars18th]

Hi @catalinii ,

I have an idea about a new extesion for minisatip, but before describe it I need to explain how to reproduce a test that illustrates the use case:

1. Create a M3U file with an HTTP link of one program of your minisatip server, for example:

#EXTM3U
#EXTINF:0,KiKA HD
http://192.168.1.35:8080/?src=1&freq=11347&pol=V&msys=DVBS2&mtype=8PSK&sr=22000&pids=0,6610,6620,6600,6610,6630

2. Open one VLC instance and open the M3U file, you'll see the service.
3. Open another VLC instance and open the same M3U file, you'll see two times the same service.
4. Now open the PlayList (CTRL + L) of one of the two VLC and click on the service. The result is that the HTTP stream is closed, and a new session is opened. And what you see in this instance is a small glitch in the playing... around 300-500ms.
5. But, now, close the second VLC instance, and execute the same procedure. What you observe? Around a cut of 2-3 seconds in the best case.

So, what's the problem that I want to illustrate? Using the HTTP streaming, the close of the socket implies every time the close of the session. And this generates some troubles with HTTP only clients, when the socket is closed by whatever reason and it wants to reconnect.

Then my suggestion is a workaround to overcome this case. The concept is based on a delayed HTTP session close. Mainly is quite simple: when one HTTP client socket is closed, the session is not closed directly, but queued. Then a timeout is started and only when it expires the session will be really closed. The timeout can be set around 5 seconds. This will enable the option for a fast HTTP reconnection. However, one case requires to be addressed. It's when the client reconnects to make a request of another different service. In this case the queued session requires to be closed inmediatly. Otherwise, the new tunning can't be served (even if it's in the same transponder). So to handle this case, you can store the IP of the client with the session in the queue, and only if this client reconnects (assuming that that the addess is not shared) and the request is different, then you stop the queued session before serve the new session.

What you think about this enhancement? My concern is with HTTP clients that doesn't support RTSP, for example Android devices (STBs or Smartphones/tablets) that work as regular IPTV clients.

[catalinii]

Let me understand the requirement:

1. client A connects to minisatip and plays the stream using http
2. client A disconnects from minisatip for whatever reason
3. client A reconnects to minisatip (let's say after 2s)

What is the expectation related to stream playing after the reconnection? To play exactly where it left off?

Basically minisatip to cache all the data for that client and send it back from the last successful socket send?

[lars18th]

Hi @catalinii,

First, I'm not suggesting a complex solution to maintain a "clean" and fully "no-lost" streaming. I think this will be complex to achieve, and then the answer to yours questions is NO.

Second, the problem is created when the adapter is closed because the HTTP socket is disconnected. This implies to close all in the backend, and when restarting it requires a lot of time to restablish it. Imagine when using the satipc module and encryption. When you close and reopen you almost need 2-3 seconds to restablish the streaming in the best cases. So that's what I want to fix.

Futhermore, after more testing past weekend, I see that the problem is in fact not with the HTTP clients. But triggered by this clients, as the close of the socket in this case every time implies the closing of the adapter. In fact, the problem is the closing of the adapter inmediatly when the call is requested. So, if the closing is delayed for almost 5 seconds, any client that will reconnect and reopens the same frequency will start to receive the streaming with zero delay.

So, the idea is this: Can you please implement some delayed closing of the adapter? Take note that "closing" implies leaving the tuner. So any "new" request sended to an adapter in a "closing" state will directly trigger the change, and will not queue it.

What you think?

[catalinii]

Hey,

Riggt now the adapter_timeout prevents the tuner to be closed for ,30s after the close of the last stream?

Is this not the case for you?

[lars18th]

Riggt now the adapter_timeout prevents the tuner to be closed for ,30s after the close of the last stream? Is this not the case for you?

I'll need to recheck it.

[lars18th]

Hi @catalinii ,

I do the check, and the adapter will be opened for the 30s. But not the session. So when using the satipc module and encryption, when the HTTP client disconnects the streaming from the SAT>IP server is stoped and the decryption too. Then when the HTTP client reconnects, even if the TCP socket restarts in 20-30ms, the session requires around 3 seconds to restablish. The behaviour is then an interruption of 3 seconds for the streaming.

Why you don't delay the closing of the session?

[catalinii]

If you refer to closing the connecting between the satip client in minisatip and the server, that was the feature you added (adapter_standby).

[lars18th]

Hi @catalinii ,

No, no. Perhaps I've not explained well. The problem is not the adapter status, but the session status! Please, execute the test described in the first post of this issue. And do it using two minisatip, one using the satipc module to drive the second (or using a hardware SAT>IP server). This test exposes what I want to explain and fix.

I try to repeat the description:

• With only 1 client connected, if this client uses the HTTP connection, when the socket is closed, this triggers the closing of the session. Then if you use the satipc module, the sesion with the remote sever is stoped and the decryption is disabled too. This is true every time, even if the client don't want to disconnect, but it occurs for a unkown reason that the HTTP socket is closed, then it's reconnected.

What it fails at time is the closing of the session. We can't change the relation "HTTP socket closed --> Session closed". And the closing of session has a very time consuming consequencies. So, it will be usefull to delay the closing of the session.

I hope it will be more clear now.

[catalinii]

I think there are 2 issues here:

• why closing/starting the session takes 2-3 seconds ...this is unexpected
• secondly, if we really need to do the session close delay...here therr are a lot of edge cases related to a new stream coming in and getting 404. If the first issue is fixed what would be acceptable reconnection times for you?

Also can u upload a log for the first case?

In general, if the tuner is not closed on the server minisatip, the pmt and cws are not deleted so a new tune should use the CWs straight away

[lars18th]

Hi @catalinii ,

why closing/starting the session takes 2-3 seconds ...this is unexpected

Because after closing the session, this triggers the closing of the satipc adapter, and this free the: a) the SAT>IP session (TEARDOWN); b) the PMT tables; c) the CWs. Then when you reconnect you need to wait until the tune is done, the streaming is started, the PMT tables are loaded, the CA/PMT is loaded, the decyption is initialized, and the CW is get. That's a lot of time. And is the expected behaviour.

In general, if the tuner is not closed on the server minisatip, the pmt and cws are not deleted so a new tune should use the CWs straight away

But the tuner IS REALLY CLOSED when the HTTP socket is closed. That's the key point! What you comment is when the tuner is not closed, but it is closed!

secondly, if we really need to do the session close delay...here therr are a lot of edge cases related to a new stream coming in and getting 404. If the first issue is fixed what would be acceptable reconnection times for you?

My suggestion of a "delayed close" is to put the session in a queue of closing-pending-sessions. So any new request will directly removes the pending close from the queue if no other tuners are free. So in fact this will be identical to a transponder change. So you don't need to trace the clients, only delay the closing of the session.

[catalinii]

@lars18th Please upload your log for tuning 2 times to the same channel (with connection close in between).

Thanks

[lars18th]

Hi @catalinii ,

Please upload your log for tuning 2 times to the same channel (with connection close in between).

This is the relevant line in the output log:

[09/03 12:12:47.665 main]: satipc: last setup less than 10 seconds ago for server 192.168.1.38, maybe an error ?

So, I isolated the problem with this part of the code: https://github.com/catalinii/minisatip/blob/dc4185b654ffd304d6873a5ae9b773c6511719af/src/satipc.c#L1219

The trouble that I have is then because: when one HTTP socket is closed and reopened very fast (less than 1 second), this part of the code blocks the opening of the new session in the remote SAT>IP server. So, first of all this part of the code requires attention. However, the root cause: closing the HTTP client session implies directly the closing of the adapter perhaps requires an enhancement too.

Regards.

[lars18th]

Hi @catalinii ,

I've created a PR that fixes this problem, in the side of the satipc module. Please, check it: #817 With it the problem is nearly solved. However, I suggest to you to improve the close of the HTTP clients. As when closing the adapter with no delay you lost a lot more packets than if you forget them but maintain the adapter open (for almost 1-2 seconds).

Regards.