H265Packet can't work properly with pion/webrtc SampleBuilder

[loveyacper]

Your environment.

github.com/pion/rtp v1.8.15 github.com/pion/webrtc/v4 v4.1.0

What did you do?

code snippet

videoSampleBuilder := samplebuilder.New(128, &H265Packet{}, 90*1000)
videoSampleBuilder.Push(rtpPacket)
for sample := pc.videoSampleBuilder.Pop(); sample != nil; sample = pc.videoSampleBuilder.Pop() {	
	log.Info("video sample  data len = %v", len(sample.Data))
}

Always print

video sample  data len = 0

What did you expect?

SampleBuilder can produce correct h265 sample.

What happened?

Because H265Packet.Unmarshal always return nil, the Sample returned by SampleBuilder always with zero length data.

[JoTurk]

Can you possibly give me details, what did you use to encode the h265 samples, what browsers, how did you get or make the RTP packets? so we can reproduce this issue.

[loveyacper]

Can you possibly give me details, what did you use to encode the h265 samples, what browsers, how did you get or make the RTP packets? so we can reproduce this issue.

Thanks for your quick reply. I think h265 encoder is irrelevant. The point is, H265Packet.Unmarshal always return nil as data, so SampleBuilder can't get any payload data, SampleBuilder.Pop() always report no error but return empty data, it's very confused.

[JoTurk]

I see, This is weird! I never used Pion's HEVC de-packetizer, I'll look into it tomorrow and I'll land a fix. Thank you.

[loveyacper]

https://github.com/pion/rtp/pull/200 This PR seems good. It does the right thing to assemble h265 nalu from RTP packets. But I already written my own H265 parser for solve problem quickly. I hope this PR can be thoroughly reviewed.

[JoTurk]

https://github.com/pion/rtp/pull/200 This PR seems good. It does the right thing to assemble h265 nalu from RTP packets. But I already written my own H265 parser for solve problem quickly. I hope this PR can be thoroughly reviewed.

No, This PR is for the safari Payloader which is not required anymore. Apple moved to the standard hevc RTP format (ios 18). I'm working on hevc save to disk example, and I'll fix all the issues with the sample builder.

[tubzby]

I encountered this bug today. It appears that H265Packet.Unmarshal() never returns data, which causes SampleBuilder to break.

[tubzby]

And, H265Packet from https://github.com/pion/rtp/pull/165 is fine for me.

[JoTurk]

I encountered this bug today. It appears that H265Packet.Unmarshal() never returns data, which causes SampleBuilder to break.

The current one uses the old design passing to Payload, but even then it's broken with some NALs, Expect the fix to land this weekend most likely.

[JoTurk]

@loveyacper hello, the empty buffer issue is fixed by v1.8.20 but it's not production ready yet we'll have to rewrite it soon, to fix all the issues.

[Sean-Der]

@cnderrauber @JoeTurki Is this ok to fix? I am not familiar with everything going on here :)

I see https://github.com/pion/rtp/commit/62819f50793022d47d15c500c2b7c6d7b75ff3db landed. What else needs to be done? I am happy to take some stuff on :)

[JoTurk]

@Sean-Der the writer needs a rewrite pretty much. if you try to use it with some complex payload with random MTU sizes you will see many issues.