bluenviron
RTPListener doesn't work
Thank you for your lib before writing issue.
i tried to make sample in window with this library and it make a successful.
so i saw your reference main.go and i setup ffmpeg in window in order to use h264decoder.go
this link is ffmpeg library (it's good with window) ffmpeg-master-latest-win64-gpl.zip
and i change the cgo flag like this
// #cgo CFLAGS: -I D:/Camera/ffmpeg-master-latest-win64-gpl-shared/include // #cgo LDFLAGS: -Ld D:/Camera/ffmpeg-master-latest-win64-gpl-shared/lib/libavcodec.dll.a // #cgo LDFLAGS: -Ld D:/Camera/ffmpeg-master-latest-win64-gpl-shared/lib/libavutil.dll.a // #cgo LDFLAGS: -Ld D:/Camera/ffmpeg-master-latest-win64-gpl-shared/lib/libswscale.dll.a // #include <libavcodec/avcodec.h> // #include <libavutil/imgutils.h> // #include <libswscale/swscale.h> import "C"
until that, it is my development environment. and from down here, i think it is issue.
with your sample, it should have worked normally. but, in my environment, RTCP callback function work well but RTP callback function doesn't called. :(
so i debuged with goland, i found the expected part. in client.go with 713 - 716 Line, it call function start (clientudpl.go) [client.go 713-716] for _, ct := range c.tracks { ct.udpRTPListener.start(true) ct.udpRTCPListener.start(true) } [clientudpl.go start] func (u *clientUDPListener) start(forPlay bool) { u.running = true u.pc.SetReadDeadline(time.Time{}) u.readerDone = make(chan struct{}) go u.runReader(forPlay) }
in this start function, it called goroutine u.runReader. I don't know the internal movement of goroutine, but it wasn't called. (only udpRTPListener.start)
so i set delay in client.go, it works well [client.go 713-716] for _, ct := range c.tracks { ct.udpRTPListener.start(true) ++ time.Sleep(time.Second * time.Duration(3)) // at least 3 second. not worked in 2 second ct.udpRTCPListener.start(true) }
The exact cause was not found, but symptoms were found so i think it seemed to be reported.
i attach my sample code and i hope it helps to resolve that Symptom.
thank you.
package main
import ( "fmt" "github.com/aler9/gortsplib" "github.com/aler9/gortsplib/pkg/base" "github.com/aler9/gortsplib/pkg/url" "image" "image/jpeg" "os" "runtime" "strconv" "time" )
func saveToFile(img image.Image) error { fname := "D://"+strconv.FormatInt(time.Now().UnixNano()/int64(time.Millisecond), 10) + ".jpg" //
f, err := os.Create(fname)
if err != nil {
fmt.Println("os.Create fname fail ", err)
os.Exit(1)
}
defer f.Close()
fmt.Println("saving", fname)
// convert to jpeg
return jpeg.Encode(f, img, &jpeg.Options{
Quality: 60,
})
}
func main() { runtime.GOMAXPROCS(8) c := gortsplib.Client{}
// parse URL
u, err := url.Parse("rtsp://myid:[email protected]:554/live4.sdp")
if err != nil {
fmt.Println("URL Parse ", err)
os.Exit(1)
}
// connect to the server
err = c.Start(u.Scheme, u.Host)
if err != nil {
fmt.Println("Start ", err)
os.Exit(1)
}
defer c.Close()
// find published tracks
fmt.Println("Describe")
tracks, baseURL, _, err := c.Describe(u)
if err != nil {
fmt.Println("Describe ", err)
os.Exit(1)
}
// find the H264 track
h264TrackID, h264track := func() (int, *gortsplib.TrackH264) {
for i, track := range tracks {
if h264track, ok := track.(*gortsplib.TrackH264); ok {
return i, h264track
}
}
return -1, nil
}()
if h264TrackID < 0 {
fmt.Println("H264 track not found")
os.Exit(1)
}
// setup H264->raw frames decoder
h264dec, err := newH264Decoder()
if err != nil {
fmt.Println("newH264Decoder ", err)
os.Exit(1)
}
defer h264dec.close()
// if present, send SPS and PPS from the SDP to the decoder
sps := h264track.SafeSPS()
if sps != nil {
h264dec.decode(sps)
}
pps := h264track.SafePPS()
if pps != nil {
h264dec.decode(pps)
}
// Callback Function
saveCount := 0
// Callback Video Udp Data
c.OnPacketRTP = func(ctx *gortsplib.ClientOnPacketRTPCtx) {
fmt.Printf("[RTP]\n%v, %v, %v\n", ctx.TrackID, ctx.H264PTS, ctx.PTSEqualsDTS) //ctx.Packet, ctx.H264NALUs, is frame Data
if ctx.TrackID != h264TrackID {
return
}
if ctx.H264NALUs == nil {
fmt.Println("H264NALUs == nil")
return
}
for _, nalu := range ctx.H264NALUs {
// convert H264 NALUs to RGBA frames
img, err := h264dec.decode(nalu)
if err != nil {
fmt.Println("h264dec.decode ", err)
os.Exit(1)
}
// wait for a frame
if img == nil {
continue
}
// ==== Save Image Start ====
err = saveToFile(img)
if err != nil {
fmt.Println("saveToFile ", err)
os.Exit(1)
}
saveCount++
if saveCount == 5 {
fmt.Println("saved 5 images Succ")
os.Exit(1)
}
// ==== Save Image End ====
//fmt.Printf("%v", img) // img data
fmt.Printf("decoded frame with size %v", img.Bounds().Max)
}
}
// Callback Rtp Check Data
c.OnPacketRTCP = func(ctx *gortsplib.ClientOnPacketRTCPCtx) {
fmt.Printf("[RTCP]\n%v, %v", ctx.TrackID, ctx.Packet)
}
// When Response msg from rtsp
c.OnResponse = func(resp *base.Response) {
fmt.Printf("%v, %v, %v, %v\n", resp.StatusMessage, resp.StatusCode, resp.Header, resp.Body)
}
// When Send smg from rtsp
c.OnRequest = func(req *base.Request) {
fmt.Printf("%v, %v, %v, %v\n", req.Method, req.URL, req.Header, req.Body)
}
// i wanna set Setup(false, t, baseURL, 0, 0) , so i didn't use SetupAndPlay Func
fmt.Println("Setup")
for _, t := range tracks {
_, err := c.Setup(true, t, baseURL, 0, 0) // i don't know why i can't set (forPlay == false) in Setup Function.
if err != nil {
fmt.Println("hihi", err)
}
}
fmt.Println("Play")
_, err = c.Play(nil)
// wait until a fatal error
panic(c.Wait())
}
//[decoder.go] // https://github.com/aler9/gortsplib/blob/main/examples/client-read-h264/h264decoder.go package main
import ( "fmt" "image" "unsafe" ) // #cgo CFLAGS: -I D:/Camera/ffmpeg-master-latest-win64-gpl-shared/include // #cgo LDFLAGS: -Ld D:/Camera/ffmpeg-master-latest-win64-gpl-shared/lib/libavcodec.dll.a // #cgo LDFLAGS: -Ld D:/Camera/ffmpeg-master-latest-win64-gpl-shared/lib/libavutil.dll.a // #cgo LDFLAGS: -Ld D:/Camera/ffmpeg-master-latest-win64-gpl-shared/lib/libswscale.dll.a // #include <libavcodec/avcodec.h> // #include <libavutil/imgutils.h> // #include <libswscale/swscale.h> import "C"
func frameData(frame *C.AVFrame) **C.uint8_t { return (**C.uint8_t)(unsafe.Pointer(&frame.data[0])) }
func frameLineSize(frame *C.AVFrame) *C.int { return (*C.int)(unsafe.Pointer(&frame.linesize[0])) }
// h264Decoder is a wrapper around ffmpeg's H264 decoder. type h264Decoder struct { codecCtx *C.AVCodecContext srcFrame *C.AVFrame swsCtx *C.struct_SwsContext dstFrame *C.AVFrame dstFramePtr []uint8 }
// newH264Decoder allocates a new h264Decoder. func newH264Decoder() (*h264Decoder, error) { codec := C.avcodec_find_decoder(C.AV_CODEC_ID_H264) if codec == nil { return nil, fmt.Errorf("avcodec_find_decoder() failed") }
codecCtx := C.avcodec_alloc_context3(codec)
if codecCtx == nil {
return nil, fmt.Errorf("avcodec_alloc_context3() failed")
}
res := C.avcodec_open2(codecCtx, codec, nil)
if res < 0 {
C.avcodec_close(codecCtx)
return nil, fmt.Errorf("avcodec_open2() failed")
}
srcFrame := C.av_frame_alloc()
if srcFrame == nil {
C.avcodec_close(codecCtx)
return nil, fmt.Errorf("av_frame_alloc() failed")
}
return &h264Decoder{
codecCtx: codecCtx,
srcFrame: srcFrame,
}, nil
}
// close closes the decoder. func (d *h264Decoder) close() { if d.dstFrame != nil { C.av_frame_free(&d.dstFrame) }
if d.swsCtx != nil {
C.sws_freeContext(d.swsCtx)
}
C.av_frame_free(&d.srcFrame)
C.avcodec_close(d.codecCtx)
}
func (d *h264Decoder) decode(nalu []byte) (image.Image, error) { nalu = append([]uint8{0x00, 0x00, 0x00, 0x01}, []uint8(nalu)...)
// send frame to decoder
var avPacket C.AVPacket
avPacket.data = (*C.uint8_t)(C.CBytes(nalu))
defer C.free(unsafe.Pointer(avPacket.data))
avPacket.size = C.int(len(nalu))
res := C.avcodec_send_packet(d.codecCtx, &avPacket)
if res < 0 {
return nil, nil
}
// receive frame if available
res = C.avcodec_receive_frame(d.codecCtx, d.srcFrame)
if res < 0 {
return nil, nil
}
// if frame size has changed, allocate needed objects
if d.dstFrame == nil || d.dstFrame.width != d.srcFrame.width || d.dstFrame.height != d.srcFrame.height {
if d.dstFrame != nil {
C.av_frame_free(&d.dstFrame)
}
if d.swsCtx != nil {
C.sws_freeContext(d.swsCtx)
}
d.dstFrame = C.av_frame_alloc()
d.dstFrame.format = C.AV_PIX_FMT_RGBA
d.dstFrame.width = d.srcFrame.width
d.dstFrame.height = d.srcFrame.height
d.dstFrame.color_range = C.AVCOL_RANGE_JPEG
res = C.av_frame_get_buffer(d.dstFrame, 1)
if res < 0 {
return nil, fmt.Errorf("av_frame_get_buffer() err")
}
d.swsCtx = C.sws_getContext(d.srcFrame.width, d.srcFrame.height, C.AV_PIX_FMT_YUV420P,
d.dstFrame.width, d.dstFrame.height, (int32)(d.dstFrame.format), C.SWS_BILINEAR, nil, nil, nil)
if d.swsCtx == nil {
return nil, fmt.Errorf("sws_getContext() err")
}
dstFrameSize := C.av_image_get_buffer_size((int32)(d.dstFrame.format), d.dstFrame.width, d.dstFrame.height, 1)
d.dstFramePtr = (*[1 << 30]uint8)(unsafe.Pointer(d.dstFrame.data[0]))[:dstFrameSize:dstFrameSize]
}
// convert frame from YUV420 to RGB
res = C.sws_scale(d.swsCtx, frameData(d.srcFrame), frameLineSize(d.srcFrame),
0, d.srcFrame.height, frameData(d.dstFrame), frameLineSize(d.dstFrame))
if res < 0 {
return nil, fmt.Errorf("sws_scale() err")
}
// embed frame into an image.Image
return &image.RGBA{
Pix: d.dstFramePtr,
Stride: 4 * (int)(d.dstFrame.width),
Rect: image.Rectangle{
Max: image.Point{(int)(d.dstFrame.width), (int)(d.dstFrame.height)},
},
}, nil
}
Hello, first of all, why did you do this?
// i wanna set Setup(false, t, baseURL, 0, 0) , so i didn't use SetupAndPlay Func
fmt.Println("Setup")
for _, t := range tracks {
_, err := c.Setup(true, t, baseURL, 0, 0) // i don't know why i can't set (forPlay == false) in Setup Function.
if err != nil {
fmt.Println("hihi", err)
}
}
Setup(false) can be called only when you're publishing a stream to a RTSP server, not when you're reading one. If you're using Describe() and Play() you're definitely reading a stream from a camera.
thanks for your response.
first i refered your example code in gortsplib/examples/client-read-h264
and during to study rtsp, i get to know that 'Setup' Request is Define the way how to send media stream. (like client port, server port)
and in my question '// i wanna set Setup(false, t, baseURL, 0, 0) , so i didn't use SetupAndPlay Func', i misunderstand first pamameters (forPay) mean it is forPlay with streaming(pulbish)
so i used that method with parameter 'true' /* _, err := c.Setup(true, t, baseURL, 0, 0) // i don't know why i can't set (forPlay == false) in Setup Function. */
and i tried to unuse Setup Method in your answer 'If you're using Describe() and Play() you're definitely reading a stream from a camera.'
in this case the function doPlay in Client.go, it returns the error cauze Preplay Error Message [ Play must be in state [prePlay], while is in state initial ] in server.go error function (67~71 line) // Error implements the error interface. func (e ErrServerInvalidState) Error() string { return fmt.Sprintf("must be in state %v, while is in state %v", e.AllowedList, e.State) }
i understand separate Setup & Play function is unnecessary so i change my code like err = c.SetupAndPlay(tracks, baseURL) if err != nil { fmt.Println("Setup And Play!") } but it doesn't work without sleep (in previous question)
thanks for your attention.
-----Original Message----- From: "Alessandro @.> To: @.>; Cc: @.>; @.>; Sent: 2022-08-23 (화) 19:46:39 (GMT+09:00) Subject: Re: [aler9/gortsplib] RTPListener doesn't work (Issue #136)
Hello, first of all, why did you do this? // i wanna set Setup(false, t, baseURL, 0, 0) , so i didn't use SetupAndPlay Func fmt.Println("Setup") for _, t := range tracks { _, err := c.Setup(true, t, baseURL, 0, 0) // i don't know why i can't set (forPlay == false) in Setup Function. if err != nil { fmt.Println("hihi", err) } } Setup(true) can be called only when you're publishing a stream to a RTSP server, not when you're reading one. If you're using Describe() and Play() you're definitely reading a stream from a camera. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>