fix timestamp for h264 packetize

[San9H0]

fix timestamp not to increase in case of sps, pps

Description

I used h264 packetizer to packetize h264 NAL data. and I set the samples values to 3000. I pushed SPS, PPS, IDR in order into the function.

I expected an increase of 3000 compared to the previous timestamp, but it increated by 9000

Reference issue

nothing

[Sean-Der]

@San9H0 Can you try pushing SPS/PPS/IDR as a single sample?

I can write a test to confirm, but I am afraid this logic could break other codecs

[San9H0]

@Sean-Der Ok. I will try

My code is simple. In my case, I try to transmit SPS and PPS before transmitting the IDR packet.

func TestPacketizer_H264_OnlyPayload(t *testing.T) {
	nonidr := []byte{0x61, 0xE0, 0x06, 0xC0, 0x0D, 0xBE, 0x47, 0x81, 0xE4, 0xF8, 0xA3, 0x88, 0xE0, 0x9C, 0xFB, 0xC4, 0x78, 0x8F, 0x11, 0xE2}
	sps := []byte{0x67, 0x42, 0xc0, 0x1f, 0x8c, 0x8d, 0x40, 0x50, 0x17, 0xfc, 0xb0, 0x0f, 0x08, 0x84, 0x6a}
	pps := []byte{0x68, 0xce, 0x3c, 0x80}
	idr := []byte{0x85, 0xB8, 0x00, 0x04, 0x00, 0x00, 0x0C, 0xE4, 0xC5, 0x00, 0x01, 0x03, 0x99, 0x33, 0x41, 0xE2, 0xD9, 0x39, 0x39}

	packetizer := NewPacketizer(1400, 98, 0x1234ABCD, &codecs.H264Payloader{}, NewRandomSequencer(), 90000)

	prevTimestamp := uint32(0)
	for _, rtpPacket := range packetizer.Packetize(nonidr, 3000) {
		fmt.Println("NON-IDR rtpPacket sn:", rtpPacket.SequenceNumber, ", ts:", rtpPacket.Timestamp, ", nalu:", rtpPacket.Payload[0]&0x1F)
		prevTimestamp = rtpPacket.Timestamp
	}

	for _, _ = range packetizer.Packetize(sps, 3000) {
		fmt.Println("SPS rtpPacket is not printed")
	}

	for _, _ = range packetizer.Packetize(pps, 3000) {
		fmt.Println("PPS rtpPacket is not printed")
	}

	for i, rtpPacket := range packetizer.Packetize(idr, 3000) {
		if i == 0 {
			fmt.Println("STAP-A rtpPacket sn:", rtpPacket.SequenceNumber, ", ts:", rtpPacket.Timestamp, ", nalu:", rtpPacket.Payload[0]&0x1F)
		}
		if i == 1 {
			fmt.Println("IDR rtpPacket sn:", rtpPacket.SequenceNumber, ", ts:", rtpPacket.Timestamp, ", nalu:", rtpPacket.Payload[0]&0x1F)
		}
		fmt.Println("this timestamp is not correct diff:", rtpPacket.Timestamp-prevTimestamp) // maybe 9000
	}
}

In the results, the timestamp`s diff is wrong.

NON-IDR rtpPacket sn: 18129 , ts: 1706210977 , nalu: 1
STAP-A rtpPacket sn: 18130 , ts: 1706219977 , nalu: 24
this timestamp is not correct diff: 9000
IDR rtpPacket sn: 18131 , ts: 1706219977 , nalu: 5
this timestamp is not correct diff: 9000

In my case, I used data without using a bitstream filter. Using your parser(H264Packet struct), I try to transmit SPS, PPS and IDR packets with Annex B added.

func TestPacketizer_H264_Payload_WithAnnexB(t *testing.T) {
	nonidr := []byte{0x00, 0x00, 0x00, 0x01, 0x61, 0xE0, 0x01, 0x40, 0x02, 0xBE, 0x40, 0x8E, 0x1C, 0x90, 0xD0, 0xBC, 0x40, 0xE8, 0x6C, 0x46, 0x78}
	spspps := []byte{0x00, 0x00, 0x00, 0x01, 0x67, 0x42, 0xC0, 0x1F, 0x8C, 0x8D, 0x40, 0x50, 0x17, 0xFC, 0xB0, 0x0F, 0x08, 0x84, 0x6A, 0x00, 0x00, 0x00, 0x01, 0x68, 0xCE, 0x3C, 0x80}
	idr := []byte{0x00, 0x00, 0x00, 0x01, 0x65, 0xB8, 0x00, 0x04, 0x00, 0x00, 0x0C, 0xE4, 0xC5, 0x00, 0x01, 0x02, 0x99, 0x3F, 0xF8, 0x7F, 0xC4, 0xEA}

	packetizer := NewPacketizer(1400, 98, 0x1234ABCD, &codecs.H264Payloader{}, NewRandomSequencer(), 90000)

	prevTimestamp := uint32(0)
	for _, rtpPacket := range packetizer.Packetize(nonidr, 3000) {
		prevTimestamp = rtpPacket.Timestamp
		fmt.Println("NON-IDR rtpPacket sn:", rtpPacket.SequenceNumber, ", ts:", rtpPacket.Timestamp, ", nalu:", rtpPacket.Payload[0]&0x1F)
	}

	for _, _ = range packetizer.Packetize(spspps, 3000) {
		fmt.Println("SPS, PPS rtpPacket is not printed")
	}

	for i, rtpPacket := range packetizer.Packetize(idr, 3000) {
		if i == 0 {
			fmt.Println("STAP-A rtpPacket sn:", rtpPacket.SequenceNumber, ", ts:", rtpPacket.Timestamp, ", nalu:", rtpPacket.Payload[0]&0x1F)
		}
		if i == 1 {
			fmt.Println("IDR rtpPacket sn:", rtpPacket.SequenceNumber, ", ts:", rtpPacket.Timestamp, ", nalu:", rtpPacket.Payload[0]&0x1F)
		}
		fmt.Println("this timestamp is not correct diff:", rtpPacket.Timestamp-prevTimestamp) // maybe 6000
	}
}

But this case, timestamp`s diff is wrong

NON-IDR rtpPacket sn: 2905 , ts: 2924337027 , nalu: 1
STAP-A rtpPacket sn: 2906 , ts: 2924343027 , nalu: 24
this timestamp is not correct diff: 6000
IDR rtpPacket sn: 2907 , ts: 2924343027 , nalu: 5
this timestamp is not correct diff: 6000

---

the other usecase is successful like this

packetizer.Packetize(append(spspps_annexB, idr_annexB...))

But I am not sure if this is correct.