go-astits
go-astits copied to clipboard
asticode
Handle truncated PES packets
Simple fix for #35
Unfortunately this is breaking some tests which are creating and operating on malformed packets.
--- FAIL: TestParsePESData (0.00s)
--- FAIL: TestParsePESData/with_header (0.00s)
data_pes_test.go:407:
Error Trace: data_pes_test.go:407
Error: Not equal:
expected: &astits.PESData{Data:[]uint8{0x64, 0x61, 0x74, 0x61}, Header:(*astits.PESHeader)(0x1381690)}
actual : &astits.PESData{Data:[]uint8{0x64, 0x61, 0x74, 0x61, 0x73, 0x74, 0x75, 0x66, 0x66}, Header:(*astits.PESHeader)(0xc0000534a0)}
Diff:
--- Expected
+++ Actual
@@ -1,4 +1,4 @@
(*astits.PESData)({
- Data: ([]uint8) (len=4) {
- 00000000 64 61 74 61 |data|
+ Data: ([]uint8) (len=9) {
+ 00000000 64 61 74 61 73 74 75 66 66 |datastuff|
},
@@ -62,3 +62,3 @@
}),
- PacketLength: (uint16) 67,
+ PacketLength: (uint16) 9,
StreamID: (uint8) 1
Test: TestParsePESData/with_header
--- FAIL: TestParseData (0.00s)
data_test.go:47:
Error Trace: data_test.go:47
Error: Not equal:
expected: []*astits.DemuxerData{(*astits.DemuxerData)(0xc000073770)}
actual : []*astits.DemuxerData{(*astits.DemuxerData)(0xc000073720)}
Diff:
--- Expected
+++ Actual
@@ -24,4 +24,4 @@
PES: (*astits.PESData)({
- Data: ([]uint8) (len=4) {
- 00000000 64 61 74 61 |data|
+ Data: ([]uint8) (len=9) {
+ 00000000 64 61 74 61 73 74 75 66 66 |datastuff|
},
@@ -85,3 +85,3 @@
}),
- PacketLength: (uint16) 67,
+ PacketLength: (uint16) 9,
StreamID: (uint8) 1
Test: TestParseData
FAIL
FAIL github.com/asticode/go-astits 0.008s
FAIL
Specifically, the with_header PES test advertises a packet length of 67 even though it only has 9 bytes of data:
https://github.com/asticode/go-astits/blob/473f66c0d7ff8e35f155c629471b214018b61fe0/data_pes_test.go#L258-L265
https://github.com/asticode/go-astits/blob/473f66c0d7ff8e35f155c629471b214018b61fe0/data_pes_test.go#L329-L332
I tried to fix these tests but they make a lot of assumptions and reuse fixtures like pesWithHeaderBytes() which creates quite the rabbit hole.
Simple fix for #35
My first attempt was more complex: https://github.com/tmm1/go-astits/commit/a55f10a084899981331ca7924bf87305c71d4e08
That approach also works, but the same tests fail for the same reason.
Coverage increased (+0.03%) to 77.24% when pulling b1ef54e2b4dc68535fcaed42f0545629dabbc87e on tmm1:truncated-pes-adjust into 473f66c0d7ff8e35f155c629471b214018b61fe0 on asticode:master.
Specifically, the with_header PES test advertises a packet length of 67 even though it only has 9 bytes of data
Are you sure this is an error? 🤔 packet length is supposed to indicate the number of bytes after this field, optional header included.
Regarding the content of the PR, I don't really like the idea of ignoring the problem and rewriting the packet length.
As you suggested here, I would implement the CorruptedPacketHandler logic and make the corrupted packet go through it.
Fixed the issue with the tests not passing.
Thanks @eric, but as I mentioned in my previous comment, I'd rather go the CorruptedPacketHandler route in this case 👍
Hi,
I'd like to share my case which might be related, I set up a muxer from a live stream into srt mpegts via astits.newmuxer and during muxing I have error messages both on go app and ffplay(receiver);
SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 185 bytes of payload: only 182 is available...
on app log and then
panic: runtime error: slice bounds out of range [1319:1316]
goroutine 12 [running]:
bufio.(*Writer).Write(0xc00029e100, {0xc0002801e1?, 0x10014c675?, 0x10014c675?}) /usr/local/go/src/bufio/bufio.go:670 +0x1c8
github.com/asticode/go-astikit.(*BitsWriter).write(0xc0002a42d0, {0xc0002801e1, 0x1, 0x1}) /Users/user/go/pkg/mod/github.com/asticode/[email protected]/binary.go:142 +0x3d
on console...
also on ffplay;
[mpegts @ 0x7fae05714180] Packet corrupt (stream = 0, dts = 6652800).
[h264 @ 0x7fae06a3d280] concealing 2720 DC, 2720 AC, 2720 MV errors in P frame
[aac @ 0x7fae068416c0] channel element 0.5 is not allocated f=0/0
I just wonder how to overcome this? What would be CorruptPacketHandler case for muxer?
Thanks in advance
AyhG
Hi @asticode ,
here is the complete dump of the log screen for the app. it'a a basic rtsp to srt gateway. test source is ffmpeg and the srt consumer is ffplay currently. I have a lot of error messages on ffplay while video is frozen before panic. Panic is not instant though, it takes a while. hope this helps.. thnak you.
the error message on ffplay is this:
...
[mpegts @ 0x7fbf19f16c00] changing packet size to 192 0B f=0/0
[mpegts @ 0x7fbf19f16c00] changing packet size to 188 0B f=0/0
[mpegts @ 0x7fbf19f16c00] changing packet size to 192 0B f=0/0
[mpegts @ 0x7fbf19f16c00] changing packet size to 188 0B f=0/0
[mpegts @ 0x7fbf19f16c00] changing packet size to 204 0B f=0/0
[mpegts @ 0x7fbf19f16c00] changing packet size to 188 0B f=0/0
1906.04 A-V:520.879 fd= 123 aq= 0KB vq= 0KB sq= 0B f=0/0
...
and the app log:
% go run .
2023/06/13 10:39:55 loading config files from config folder ./config!...
2023/06/13 10:39:55 Running RTSP Client!... live2
2023/06/13 10:39:55 Running RTSP Client!... live1
2023/06/13 10:39:55 Socket connected!... &{129186975 false 0xc000236090 127.0.0.1 5808 map[blocking:0 latency:250 mode:caller payloadsize:1316 streamid:CAM1 transtype:live] 2 1456 -1}
2023/06/13 10:39:55 Socket connected!... &{129186974 false 0xc0002b0090 127.0.0.1 5809 map[blocking:0 latency:250 mode:caller payloadsize:1316 streamid:CAM2 transtype:live] 2 1456 -1}
2023/06/13 10:39:55 IDR Not Present Error:
2023/06/13 10:39:55 Encoding video access units into muxer error: IDR Not Present Error...
2023/06/13 10:39:55 IDR Not Present Error:
2023/06/13 10:39:55 Encoding video access units into muxer error: IDR Not Present Error...
2023/06/13 10:39:55 IDR Not Present Error:
2023/06/13 10:39:55 Encoding video access units into muxer error: IDR Not Present Error...
...
2023/06/13 10:40:09 IDR Not Present Error:
2023/06/13 10:40:09 Encoding video access units into muxer error: IDR Not Present Error...
2023/06/13 10:40:09 IDR Not Present Error:
2023/06/13 10:40:09 Encoding video access units into muxer error: IDR Not Present Error...
2023/06/13 10:40:16 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 272 bytes of payload: only 182 is available...
2023/06/13 10:40:16 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:40:16 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:40:16 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 293 bytes of payload: only 182 is available...
2023/06/13 10:40:16 SRT Audio ErrorChannel active... 1002
2023/06/13 10:40:16 Encoding audio access units into muxer error: SRT Send Error...
2023/06/13 10:40:44 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 333 bytes of payload: only 182 is available...
2023/06/13 10:40:44 SRT Audio ErrorChannel active... 1002
2023/06/13 10:40:44 Encoding audio access units into muxer error: SRT Send Error...
2023/06/13 10:42:04 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 356 bytes of payload: only 184 is available...
2023/06/13 10:42:04 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:42:04 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:42:14 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 187 bytes of payload: only 182 is available...
2023/06/13 10:42:14 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:42:14 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:42:14 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 187 bytes of payload: only 182 is available...
2023/06/13 10:42:14 SRT Audio ErrorChannel active... 1001
2023/06/13 10:42:14 Encoding audio access units into muxer error: SRT Send Error...
2023/06/13 10:43:08 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 183 bytes of payload: only 182 is available...
2023/06/13 10:43:08 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:43:08 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:43:19 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 193 bytes of payload: only 182 is available...
2023/06/13 10:43:19 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:43:19 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:43:45 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 143 bytes of payload: only 140 is available...
2023/06/13 10:43:45 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:43:45 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:43:55 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 236 bytes of payload: only 184 is available...
2023/06/13 10:43:55 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:43:55 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 350 bytes of payload: only 182 is available...
2023/06/13 10:43:55 SRT Audio ErrorChannel active... 1002
2023/06/13 10:43:55 Encoding audio access units into muxer error: SRT Send Error...
2023/06/13 10:43:55 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:44:17 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 321 bytes of payload: only 184 is available...
2023/06/13 10:44:17 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:44:17 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:44:34 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 310 bytes of payload: only 182 is available...
2023/06/13 10:44:34 SRT Audio ErrorChannel active... 1002
2023/06/13 10:44:34 Encoding audio access units into muxer error: SRT Send Error...
2023/06/13 10:44:34 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 305 bytes of payload: only 182 is available...
2023/06/13 10:44:34 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:44:34 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:44:39 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 197 bytes of payload: only 182 is available...
2023/06/13 10:44:39 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:44:39 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:44:39 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 55 bytes of payload: only 30 is available...
2023/06/13 10:44:39 SRT Audio ErrorChannel active... 1001
2023/06/13 10:44:39 Encoding audio access units into muxer error: SRT Send Error...
2023/06/13 10:44:39 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 258 bytes of payload: only 182 is available...
2023/06/13 10:44:39 SRT Audio ErrorChannel active... 1002
2023/06/13 10:44:39 Encoding audio access units into muxer error: SRT Send Error...
2023/06/13 10:45:03 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 229 bytes of payload: only 184 is available...
2023/06/13 10:45:03 SRT Audio ErrorChannel active... 1002
2023/06/13 10:45:03 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 343 bytes of payload: only 182 is available...
2023/06/13 10:45:03 Encoding audio access units into muxer error: SRT Send Error...
2023/06/13 10:45:03 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:45:03 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:45:13 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 245 bytes of payload: only 184 is available...
2023/06/13 10:45:13 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:45:13 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:46:08 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 219 bytes of payload: only 184 is available...
2023/06/13 10:46:08 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:46:08 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:46:23 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 213 bytes of payload: only 182 is available...
2023/06/13 10:46:23 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:46:23 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:46:55 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 222 bytes of payload: only 182 is available...
2023/06/13 10:46:55 SRT Audio ErrorChannel active... 1002
2023/06/13 10:46:55 Encoding audio access units into muxer error: SRT Send Error...
2023/06/13 10:46:55 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 142 bytes of payload: only 68 is available...
2023/06/13 10:46:55 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:46:55 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:47:48 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 185 bytes of payload: only 184 is available...
2023/06/13 10:47:48 SRT Audio ErrorChannel active... 1001
2023/06/13 10:47:48 Encoding audio access units into muxer error: SRT Send Error...
2023/06/13 10:48:44 SRT MPEGTS muxer write SRT Video error: writePacket: can't write 156 bytes of payload: only 153 is available...
2023/06/13 10:48:44 SRT Video ErrorChannel active... 127.0.0.1
2023/06/13 10:48:44 Encoding video access units into muxer error: SRT Send Error...
2023/06/13 10:48:44 SRT MPEGTS muxer write SRT Audio error: writePacket: can't write 175 bytes of payload: only 22 is available...
2023/06/13 10:48:44 SRT Audio ErrorChannel active... 1002
2023/06/13 10:48:44 Encoding audio access units into muxer error: SRT Send Error...
panic: runtime error: slice bounds out of range [120:0]
goroutine 52 [running]:
bytes.(*Buffer).Write(0x3?, {0xc000280318?, 0xc00004a970?, 0x10039f53b?})
/usr/local/go/src/bytes/buffer.go:172 +0xd6
github.com/asticode/go-astikit.(*BitsWriter).write(0xc000298320, {0xc000280318, 0x1, 0x1})
/Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/binary.go:142 +0x3d
github.com/asticode/go-astikit.(*BitsWriter).flushBsCache(...)
/Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/binary.go:169
github.com/asticode/go-astikit.(*BitsWriter).writeFullByte(0x0?, 0x20?)
/Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/binary.go:179 +0x7e
github.com/asticode/go-astikit.(*BitsWriter).writeByteSlice(0xc000298320, {0xc000599180, 0xa8, 0xc00004aaa0?})
/Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/binary.go:130 +0x89
github.com/asticode/go-astikit.(*BitsWriter).Write(0xc000298320?, {0x1003e5cc0?, 0xc00004aa50?})
/Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/binary.go:75 +0xa5
github.com/asticode/go-astits.writePESData(0x100405d00?, 0xc000282570?, {0xc000599180?, 0x1aa?, 0x1aa?}, 0x8?, 0xb6?)
/Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/data_pes.go:465 +0xb3
github.com/asticode/go-astits.(*Muxer).WriteData(0xc0002846e0, 0xc00004acb8)
/Users/anyuser/go/pkg/mod/github.com/asticode/[email protected]/muxer.go:230 +0x4dd
X.7/gateway.(*SRT).Encode(0xc0000e2580, {0xc0003c9aa0, 0x4, 0x10038d28a?}, {0x0?, 0x10005687f?, 0x10004e552?}, 0x7db35b0a00)
/Users/anyuser/GolandProjects/X.7/gateway/srt.go:175 +0x10cd
X.7/gateway.(*Gateway).RunRTSP.func1(0xc000292140?)
/Users/anyuser/GolandProjects/X.7/gateway/rtsp.go:127 +0x191
github.com/bluenviron/gortsplib/v3.(*clientFormat).readRTPUDP(0xc0002983c0, 0xc000282870?)
/Users/anyuser/go/pkg/mod/github.com/bluenviron/gortsplib/[email protected]/client_format.go:117 +0x185
github.com/bluenviron/gortsplib/v3.(*clientMedia).readRTPUDPPlay(0xc000296480, {0xc000176000, 0xc00004af20?, 0x5c1})
/Users/anyuser/go/pkg/mod/github.com/bluenviron/gortsplib/[email protected]/client_media.go:291 +0xae
github.com/bluenviron/gortsplib/v3.(*clientUDPListener).runReader(0xc0002b80e0)
/Users/anyuser/go/pkg/mod/github.com/bluenviron/gortsplib/[email protected]/client_udp_listener.go:195 +0x1e5
created by github.com/bluenviron/gortsplib/v3.(*clientUDPListener).start
/Users/anyuser/go/pkg/mod/github.com/bluenviron/gortsplib/[email protected]/client_udp_listener.go:153 +0xaa
exit status 2
Can you open an issue including test data which would allow us to reproduce the problem so that it could be traced more easily?