pythonarcade
Video Capture in Arcade
I created a shader filter and used it with a video as an input channel, it is working properly but I want to save the rendered frames and make a video out of it.
Code:
import arcade
from arcade.experimental.shadertoy import Shadertoy
import cv2
SCREEN_WIDTH = 400
SCREEN_HEIGHT = 300
SCREEN_TITLE = "ShaderToy Video"
class ShadertoyVideo(arcade.Window):
def __init__(self, width, height, title):
super().__init__(width, height, title, resizable=True)
self.shadertoy = Shadertoy(
self.get_framebuffer_size(),
"""
void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
// Normalized pixel coordinates (from 0 to 1)
vec2 suv = fragCoord/iResolution.xy;
fragColor = vec4(1.5 * sin(suv.y * iResolution.y/3. + iTime * 20.));
fragColor = 1.- floor(abs(fragColor));
fragColor *= vec4(sin(suv.y), 0, cos( 1. - suv.y * 2.) , 1);
fragColor *= texture(iChannel0, suv);
}
""",
)
# INSERT YOUR OWN VIDEO HERE
self.video = cv2.VideoCapture("Example.mp4")
width, height = (
int(self.video.get(cv2.CAP_PROP_FRAME_WIDTH)),
int(self.video.get(cv2.CAP_PROP_FRAME_HEIGHT)),
)
self.video_texture = self.ctx.texture((width, height), components=3)
self.video_texture.wrap_x = self.ctx.CLAMP_TO_EDGE
self.video_texture.wrap_y = self.ctx.CLAMP_TO_EDGE
self.video_texture.swizzle = "BGR1"
self.shadertoy.channel_0 = self.video_texture
self.set_size(width, height)
def on_draw(self):
self.clear()
self.shadertoy.render()
def on_update(self, delta_time: float):
self.shadertoy.time += delta_time
self.next_frame()
def on_resize(self, width: float, height: float):
super().on_resize(width, height)
self.shadertoy.resize(self.get_framebuffer_size())
def next_frame(self):
exists, frame = self.video.read()
frame = cv2.flip(frame, 0)
if exists:
self.video_texture.write(frame)
if __name__ == "__main__":
ShadertoyVideo(SCREEN_WIDTH, SCREEN_HEIGHT, SCREEN_TITLE)
arcade.run()
Is there any way to save the new rendered frames and make a video? Please help :)
There is a simple capture module on moderngl-window we thought about porting into arcade. It simply pipes raw frames in stdin : https://github.com/moderngl/moderngl-window/tree/master/moderngl_window/capture
You can probably easily adapt that one to arcade.
I am still confused on what to do :(, can't we just get the frames by any method and use opencv to convert it?
I guess you could check out the VideoWriter in opencv, but I haven't used that before.
To access the raw pixel data from the window framebuffer you can window.ctx.screen.read() to get the byte data.
@einarf Ya, I know about the VideoWriter method of opencv, but we have to get the frame data of the window one by one to write the video.
You know how to get the frame data now. Be free to ask questions if you are stuck. We also have a discord server that might be a better place to have conversations about this. The plan was to add a simple capture module in arcade 2.7 at some point, so maybe this can help getting the work done.
@einarf I have used this method cv2.imwrite("out.png", self.ctx.screen.read()) to write the image file but this is giving errors.
I think opencv wants a numpy array with a certain shape and data type. When asking for help you should always include the actual error.
Again: Discord is better for this if possible
