Discussion: handle slow peers (including HTTP peers)

[Chocobozzz]

Hi :)

Original issue from https://github.com/Chocobozzz/PeerTube/issues/2047

If we try to download a segment from a P2P peer or from a CDN, but if the remote peer is slow, it's difficult to detect, abort the download and try another peer. I don't think adding another variable to timeout a download (P2P download or HTTP download) would be a good idea, because maybe it's just the client that is slow (and we would abort all its downloads).

I don't really know how we could detect that a remote peer is slow, but I have some clues:

• Compute the average bandwidth per peer and store the max:
  • If this maximum average bandwidth is greater than the segment bitrate then abort the downloads when the remote peer has an average bandwidth < segment bitrate
  • It the maximum average bandwidth is not greater than the segment bitrates then add a percentage API options on which we abort a download regarding the max average bandwidth available. Example: I download a segment from a peer (1.example.com) at 2MB/s. If I set the option to 50%, and I download a segment from another peer (2.example.com) but the average bandwidth is 700KB/s then I abort the download (1MB/s would be the threshold). We could also use this setting even if the maximum average bandwidth is not greater than the segment bitrates, to eject too slow remote peers
• Blacklist remote ejected P2P peers

What do you think? Do you have a better idea?

[ghost]

Hi! Thanks for opening this issue. I'm wondering if there's a way to specifically prevent video from pausing, by retrying downloads far enough in advance in order to prevent playback from stalling. This would address the main symptom I'm observing. There might also be ways to change how the different peers are weighted relative to each other but I don't know how this currently works :)

[mrlika]

There is requiredSegmentsPriority configuration parameter. If a segment is close enough to the playhead and still not downloaded - P2P download, if any, will be canceled and HTTP download started.

[Chocobozzz]

@mrlika In our use case a peer could be a P2P or a HTTP peer, so this configuration parameter does not help.

[ckcr4lyf]

What is an HTTP "peer"? Do you mean a peer using HTTP as the source, and the HTTP download itself is slow?

[Chocobozzz]

Do you mean a peer using HTTP as the source, and the HTTP download itself is slow?

Yes, a HTTP peer is for example a CDN, or a static mirror. Our use case could be simplified like that: imagine you have a video file and x URLs that serve this video file. We use segmentUrlBuilder so p2p-media-loader downloads segments from different endpoints. But sometimes, an endpoint is slow and we need to know that, to abort the HTTP download and try with another URL.

[ckcr4lyf]

Ah, so multiple HTTP sources. I believe in that case an extra parameter would be needed, allowing for the "client" to judge in realtime which "servers" or "http peers" have good performance and rate them as such. Bandwidth could be one metric but perhaps something else as well.

[ogmkp]

If we encounter a lot of slow video loading with webtorrent, do you think we have to disable the tracker in the static configuration while we find a solution here ?

[ckcr4lyf]

@maxdrash is your problem related to this issue? If "slow video loading in webtorrent" you mean static videos shared via webtorrent this isn't the right place.

I'm also unable to understand why disabling tracker would help?

If you mean the p2p peers are slow and so cannot watch the video, you can increase the probability of HTTP download.

[mrlika]

If P2P is slow it switches to HTTP. It should not download slowly than from HTTP.

[Chocobozzz]

If P2P is slow it switches to HTTP. It should not download slowly than from HTTP.

Unfortunately, it could be slow from HTTP too. Not everybody pay a CDN. You could also have a network of mirrors, where some of them are slow.

[IntBoat]

so...how to change the segment url (to another server) when http download is too slow?

[mrlika]

It is not a standard feature of HLS/MPEG-DASH video streaming but I think it can be implemented using the configuration parameter:

segmentUrlBuilder	Function	undefined	Allow to modify the segment URL before HTTP request. If handled, expected a function of one argument of type Segment that returns a string - generated segment URL.

https://github.com/Novage/p2p-media-loader/tree/master/p2p-media-loader-core#hybridloader

[Chocobozzz]

Hello,

This issue is still useful in our use case