Real time video streaming

[Alex-Wenner-FHR]

Search before asking

• [X] I have searched the Supervision issues and found no similar feature requests.

Question

I have a question around the VideoSink. I know that support for a static video file is there, but is there support for real time streaming? Maybe by using something like an RTSP address?

I was digging through the docs and did not find anything that insinuated there was.

Thanks!

Additional

No response

[github-actions[bot]]

Hello there, thank you for opening an Issue ! 🙏🏻 The team was notified and they will get back to you asap.

[hardikdava]

@Alex-Wenner-FHR Thanks for raising an issue. Currently, it is not supported. If cv2.VideoCapture supports direct RTSP stream then you should be able to use it directly. Otherwise, I am afraid it is not supported yet.

[SkalskiP]

Hi @Alex-Wenner-FHR 👋🏻 Like @hardikdava said, it is not supported yet. But I like the idea a lot, I'd love to add this to our roadmap.

[Alex-Wenner-FHR]

Would you be willing to provide a tiny sample involving using the cv2.VideoCapture instead of the VideoSink?

No big deal if not - just curious to see how you would implement this combination of supervision and opencv!

Feel free to close this comment whenever you see fit 😄

[hardikdava]

@Alex-Wenner-FHR after a quick search I found this. But as I mentioned, we can not help you at the moment.

[Alex-Wenner-FHR]

@hardikdava thanks! I know that this is possible with cv2. I am just curious as to how to combine this with supervision code itself. No big deal though - I can work through it.

I would love to see this feature implemented so I can stay on Supervision's level of abstraction! Thanks again.

[hardikdava]

You can do as following:

import supervision as sv

video_info = VideoInfo(width=3840, height=2160, fps=25)

with sv.VideoSink(target_path='target_video.mp4', video_info=video_info) as sink:
    YOUR PART OF FRAME GENERATION
    sink.write_frame(frame=frame)

As of directly capturing RTSP stream with sv.get_video_frames_generator() is not supported yet.

@SkalskiP we could allow to use sv.VideoSink() with callback method.

[spencerswagger]

you can edit local source code in supervision/util/video.py.

149:  if not success or frame_position >= end > 0:

I added > 0, because RTSP stream total frames always less than 0.

!!! It's only for local develoment, i'm not sure if it is stable.

[SkalskiP]

Hi @Alex-Wenner-FHR and @spencerswagger 👋🏻 I'm sorry for such a late response. I've been on vacation for the past two weeks and now I'm trying to catch up.

Would something like this work?

import supervision as sv

video_info = VideoInfo(width=3840, height=2160, fps=25)

with sv.VideoSink(target_path='target_video.mp4', video_info=video_info) as sink:
    
    cap = cv2.VideoCapture(RTSP_ADDRESS)

    while True:
        ret, frame = cap.read()
        sink.write_frame(frame=frame)

[cometothed4rkside]

Hello, how can we do this for an m3u8 url?

[onuralpszr]

Hello, how can we do this for an m3u8 url?

Hello yes opencv already handle m3u8 streams as well so all good

[iraadit]

Hi

I would also be interested in video streaming, specifically by a 'process_stream()' function to complement 'process_video()'

[wilsonlv]

I am also paying attention to this issue, does it work now ?

[SkalskiP]

Hi @wilsonlv 👋🏻 Are you interested in sending video streams or capturing them?

[wilsonlv]

@SkalskiP capture them and then track object, now i think i have worked it out, thank you all the same !

[iraadit]

How did you do @wilsonlv ?

[wilsonlv]

@iraadit First, capture the RTSP video stream using CV, and then just hand each frame over to SV for processing.

[SkalskiP]

@wilsonlv and @iraadit, take a look at my last YT video, where I showed how to process video streams in real-time using supervision and inference. I think it is exactly what you are looking for. We also shared the code. I'm closing the issue, but feel free to keep discussing it if you have more questions.

[g7crservice-harshita-gupta]

Hello All, I am facing an issue where my RTSP streaming get stopped after 5 to 10 minutes. It will be great if some help can be provided. I invested enough about this but did not found any solution.

RTSP camera Specification: https://signellent.com/uploads/media/HIKVISIONDS-2CD130P-I.pdf This error logged mostly: [h264 @ 0x55760b366d40] error while decoding MB 86 9, bytestream -14 [h264 @ 0x55760b366d40] left block unavailable for requested intra4x4 mode -1 [h264 @ 0x55760b366d40] error while decoding MB 0 6, bytestream 47281 [h264 @ 0x55760b366d40] error while decoding MB 37 65, bytestream -8

I am using cv2 package: cap = cv2.VideoCapture(url)

import time

#Replace with your RTSP URL
rtsp_url = 'xxxxx'

def connect_to_stream(url):
    cap = cv2.VideoCapture(url)
    start_time = time.time()
    return cap, start_time

def read_frame(cap):
    ret, frame = cap.read()
    return ret, frame

def reconnect_stream(url):
    print("Attempting to reconnect...")
    cap.release()
    time.sleep(5)  # Wait for 5 seconds before trying to reconnect
    cap, start_time = connect_to_stream(url)
    return cap, start_time

cap, start_time = connect_to_stream(rtsp_url)

if not cap.isOpened():
    print("Error: Could not open video stream")
else:
    while True:
        ret, frame = read_frame(cap)
        
        if not ret:
            end_time = time.time()
            total_run_time = end_time - start_time
            print(f"Stream ran for {total_run_time} seconds, exited with Error: Could not read frame")
            cap, start_time = reconnect_stream(rtsp_url)
            continue  # Continue to the next iteration of the loop
        gray_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        cv2.imshow('RTSP Stream (Grayscale)', gray_frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    cap.release()
    cv2.destroyAllWindows()

[LinasKo]

@PawelPeczek-Roboflow, any insights on this?