PyWebIO
PyWebIO copied to clipboard
pywebio
[feature request] Output method to display video in browser
I am making a computer vision app where I need to display the webcam stream on the browser. Is there a way to do this currently in pywebio?
For example, in flask, we can do it this way. But, of course, since quite a lot of effort is needed to achieve this functionality in flask, I would very much love it if pywebio can handle this. Something like 'put_video', similar to 'put_image' would be great!
Video streaming is a very interesting use case, however, the tricky part is how do you handle your back-end content delivery network? It's very challenging to globally deliver high-quality live streaming services over the internet. I think a workaround can be using a 3rd party stream API or service, then embedding an HTML block using put_html() in your app.
Thoughts?
Interesting idea. This is what I tried just now:
import cv2
from pywebio.output import put_html
cap = cv2.VideoCapture(0)
while True:
if cap.grab():
flag, frame = cap.retrieve()
if not flag:
continue
else:
cv2.imwrite("image.png", frame)
html = "<html><img src='image.png'></html"
put_html(html)
Its a simple webcam stream. And the idea is to save each frame in the video stream as an image, convert the image into an html object, and use put_html() to display the output in browser. But it produces blank images, and also seems to try to display the images next to each other instead of replacing the previous image with the current one (to give the appearance of a video). Any ideas on how to make it work?
I also tried saving each image as an html file, and then reading that html file, but I get the same output:
...
...
else:
cv2.imwrite("image.png", frame)
html_str = "<html><img src='image.png'></html"
Html_file= open("filename.html","w")
Html_file.write(html_str)
Html_file.close()
with open("filename.html", 'r') as f:
html = f.read()
put_html(html)
I also tried displaying each image directly using put_image(), and it displays each image one below the other, it doesn't replace the previous image with the current one:
import cv2
from pywebio.output import put_image
cap = cv2.VideoCapture('path/to/video.mp4') # or leave as 0 for webcam
while True:
if cap.grab():
flag, frame = cap.retrieve()
if not flag:
continue
else:
cv2.imwrite("image.png", frame)
put_image(open('image.png', 'rb').read())
I tried use_scope() (https://pywebio.readthedocs.io/en/latest/guide.html#use-scope) and time.sleep() to only keep one image at a time. The caveat of this workaround is the fade-in effect upon clearing a scope.
import cv2
from pywebio.output import *
import time
cap = cv2.VideoCapture(0)
while True:
if cap.grab():
flag, frame = cap.read()
if not flag:
continue
else:
cv2.imwrite("image.png", frame)
img = open("image.png", 'rb').read()
#put_image(img)
with use_scope('stream', clear=True):
put_image(img)
time.sleep(0.5)
@helincao
The caveat of this workaround is the fade-in effect upon clearing a scope.
The fade-in effect can be turn off by pywebio.session.set_env(output_animation=False)
The fade-in effect can be turn off by
pywebio.session.set_env(output_animation=False)
Thanks for this suggestion. Where exactly in the code should this line be? I put it at the end, and it doesn't do anything.
The fade-in effect can be turn off by
pywebio.session.set_env(output_animation=False)Thanks for this suggestion. Where exactly in the code should this line be? I put it at the end, and it doesn't do anything.
@PrashantSaikia You should put the set_env() at the beginning of your code.
Indeed, there is file reading/writing latency that prevents your "streaming" from being smooth and yields this flicker effect. This is pretty much how far you can get using the currently available PyWebIO modules. A dedicated module for streaming output is not a trivial task, and we do not have a plan to support it by the end of 2021 yet. However, things can change if the demand for this module becomes high :)
is there any road maps which features will be added on in the curse of time? @helincao
It's not there at this moment. An open roadmap or to-do list might be an interesting idea. Let us do a little exploration and see how to support it. If you have ideas, please join our Discord server for brainstorming: https://discord.gg/MvaCcg76Z7