Support hls.js v1

[mrlika]

Add hls.js v1 support to P2P Media Loader hls.js integration

Issues:

• [x] ~~Manually changing bitrate hangs playback~~
• [x] ~~Estimated bitrate and selected level is too low~~
• [ ] Improve bandwidth estimation

[Chocobozzz]

From what I saw the estimated bandwidth is accurate after some seconds, but if we are on the edge of the steam, HLS will just load segments at the playlist bitrate. Then our estimated bitrate will be caped at the playlist bitrate.

I think that instead of guessing stats.loading.{start,end,first} we should add in the segment startDownloadTime and endDownloadTime attributes. This way hls.js would have the actual download bandwidth per segment instead of download bandwidth per interval (that depends on how the buffer is full or not).

[mrlika]

@Chocobozzz, I hard-coded bandwidth to be always 10mbps in core but still chosen by hls.js quality is too low. Investigating...

[mrlika]

The reason is that now hls.js relies on stats while a segment is still loading. _abandonRulesCheck once download time is greater than 1/2 of segment duration tries to abandon segment based on these real-time stats.

P2P Media Loader doesn't provide stats updates while a segment is loading that is why it gets abandoned.

The solution is to populate stats.bwEstimate field with a value that is an estimate of the current P2P network bandwidth.

Working on the fix.

[mrlika]

Please test. It should work stable but the bandwidth estimator requires improvement. Thinking about how to do it.

[kontrollanten]

I've no deeper knowledge or understand in how p2p-media-loader works, but is there anything certain that I can help to test? I'm afraid that teaching me how to test may be harder than to do the actual testing, anyway I'm here to help if I can. :)

[Chocobozzz]

@mrlika Hi, how do you plan to improve the bandwidth estimator?

[Chocobozzz]

@mrlika Are the improvements of the bandwidth estimator mandatory for hls v1 support?

[Chocobozzz]

I created a fork for peertube using this branch and adding additional commits: https://github.com/chocobozzz/p2p-media-loader/tree/peertube

I won't make a PR because this branch only works with hls v1 and adds breaking changes in some events but basically it:

• Fix the bandwidth approximator (see https://github.com/Novage/p2p-media-loader/pull/192#issuecomment-891846824)
• core emits segment-start-load event to notify when it starts to download a segment
• core emits a segment-size event when it knows the size of the segment
• bytes-downloaded and bytes-uploaded emit the segment in parameters
• hls loader correctly set segment loading.start and total stats using segment-start-load and segment-size events
• hls loader correctly set loaded stats using bytes-downloaded event
• hls loader does not provide bandwidth stats to HLS, since HLS already has a good approximation of the current bandwidth using the fragment stats (loading.start and loaded)
• hls loader creates fake stats for segments that are not downloaded by HTTP so the HLS ABR waits for HTTP download (we may are trying to download a low priority segment using P2P)

See https://github.com/Chocobozzz/p2p-media-loader/commit/1f84f463a3556d856c1ed40b12347d358cd4b158

[MasterEvilAce]

Any movement on this?

[juanpablocs]

what is status of this?

[MasterEvilAce]

• bytes-downloaded and bytes-uploaded emit the segment in parameters

See Chocobozzz@1f84f46

Thanks for this. I've switched over to your branch for the time being since it may take a while to get this one updated, and everything seems to work fine from what I can tell, except that listening to the engine.on("piece_bytes_uploaded"...) event never triggers. piece_bytes_downloaded works as expected. Infact, I can see via the downloaded event that method=p2p is happening... so data is transferring between peers. It's just that the uploaded event never triggers. Seems to work fine if I swap back to Novage's version.

[Chocobozzz]

Thanks, it should be fixed by https://github.com/Chocobozzz/p2p-media-loader/commit/b51e6a1e775eeb56698789bef3f3336d4b72ba5f

[MasterEvilAce]

Update doesn't seem to change anything. Just to confirm, here are the packages I'm using and the line that should be expected to be called. The hls/p2p engine setup is pretty standard from the existing documentation.

<script src="https://cdn.jsdelivr.net/npm/@peertube/[email protected]/build/p2p-media-loader-core.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@peertube/[email protected]/build/p2p-media-loader-hlsjs.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/hls.min.js"></script>
... 

engine.on("piece_bytes_uploaded", (method, ...args) => {
	console.log('uploaded');
});

[radiantmediaplayer]

@Chocobozzz I just tried your last commits at https://github.com/chocobozzz/p2p-media-loader/tree/peertube (thanks for that - your fork is very helpful) but like @MasterEvilAce I am still not able to register to p2pml.core.Events.PieceBytesUploaded events. In the below code p2pml.core.Events.PieceBytesUploaded will never fire. Is it something you can look into?

        this.engine.on(p2pml.core.Events.PieceBytesDownloaded, (method, segment, bytes) => {
          window.console.log('p2pml.core.Events.PieceBytesDownloaded');
          this.downloadStats.push({ method: method, size: bytes, timestamp: performance.now() });
          this.downloadTotals[method] += bytes;

        });
        this.engine.on(p2pml.core.Events.PieceBytesUploaded, (method, segment, bytes) => {
          window.console.log('p2pml.core.Events.PieceBytesUploaded');
          this.uploadStats.push({ size: bytes, timestamp: performance.now() });
          this.uploadTotal += bytes;
        });

[MasterEvilAce]

@radiantmediaplayer I attempted to debug this a bit ago and found that the event was just not being propagated all the way "up" through the objects. I even re-implemented the function that choco changed in their original fix (outside of the package, just rewiring up all the object and event calls), and it works when used externally. So honestly I'm not sure why it's not firing. I suspect it might be an async/await promise issue or some such. /shrug

I use this work around. I'm not sure if it's perfect, or if there's an easier way... but it seems to work! Basically when a peer connects, we then listen for the bytes-uploaded event on the peer itself, rather than trying to listen on the engine object.

window.p2p.on(p2pml.core.Events.PeerConnect, (peerInfo) => { const peer = window.p2p.loader.p2pManager.peers.get(peerInfo.id); peer.on('bytes-uploaded', (peer, segmentId, bytes) => { stat_bytes_upload += bytes; }); });

EDIT: I realize I should specify that window.p2p is the same as just "engine".

[Chocobozzz]

Hi,

I think it was a build issue. I republished a version on NPM that should include my fixes (@peertube/[email protected]).

[MasterEvilAce]

Weird. Still no change in upload event functionality. Also, hlsjs loader does not work without core loader. p2p-media-loader-hlsjs.min.js:1 Uncaught Error: Cannot find module 'events' at s (p2p-media-loader-hlsjs.min.js:1:155) at p2p-media-loader-hlsjs.min.js:1:296 at Object.require.3../hlsjs-loader (p2p-media-loader-hlsjs.min.js:1:1790) at s (p2p-media-loader-hlsjs.min.js:1:261) at p2p-media-loader-hlsjs.min.js:1:296 at Object.require.p2p-media-loader-hlsjs../engine (p2p-media-loader-hlsjs.min.js:45:1121) at s (p2p-media-loader-hlsjs.min.js:1:261) at p2p-media-loader-hlsjs.min.js:1:296 at Object.require.1../index (p2p-media-loader-hlsjs.min.js:1:1072) at s (p2p-media-loader-hlsjs.min.js:1:261)

EDIT: Looks like upload event started working today, even though I confirmed it was using 1.0.9 yesterday. I am not sure what the deal is.

[radiantmediaplayer]

@Chocobozzz thanks for the npm new version. I can see PieceBytesUploaded events now.

[jazz1611]

hello, can you suggest the setting for VOD with 5sec per segment in JW Player? I use default setting or below setting but it almost stuck at 9-14 sec in duration or more 1 min duration video. my manifest is HLS with multi quality on it.

  var engine = new p2pml.hlsjs.Engine({
          segments: {
              swarmId: ID,
          },
          loader: {
              trackerAnnounce: ["wss://customtracker", "wss://tracker.openwebtorrent.com"],
              cachedSegmentExpiration:86400000,
              cachedSegmentsCount:1000,
	}
  });

or

  var engine = new p2pml.hlsjs.Engine({
          segments: {
              // forwardSegmentCount: 20,
              swarmId: ID,
          },
          loader: {
              trackerAnnounce: ["wss://customtracker", "wss://tracker.openwebtorrent.com"],
              cachedSegmentExpiration:86400000,
              cachedSegmentsCount:1000,
              requiredSegmentsPriority:1,
              httpDownloadMaxPriority:9,
              httpDownloadProbability:0.06,
              httpDownloadProbabilityInterval: 1000,
              httpDownloadProbabilitySkipIfNoPeers: true,
              p2pDownloadMaxPriority: 50,
              httpFailedSegmentTimeout:1000,
              simultaneousP2PDownloads:20,
              simultaneousHttpDownloads:3,
              httpDownloadInitialTimeout: 60000,
              httpDownloadInitialTimeoutPerSegment: 3000,
              httpUseRanges: true,
	}
  });

[hktrace]

@Chocobozzz I found that when loading the hls data the .ts files are not loaded in the order of the files in m3u8 in the browser. Instead, it is random. Do you know how to fix this problem? Under normal circumstances 0.ts 1.ts 2.ts 3.ts 4.ts 5.ts However, when loading within this project, it is like this 0.ts 1.ts 5.ts 3.ts. 2.ts 4.ts