Feature request: Video support for macroquad

[LeonStansfield]

I would like to request a feature that would allow macroquad to support video playback. This would be helpful for use developing visual related software with macroquad. For example, in my project TawnyAV, I would like to be able to load a pre rendered video and apply post processing effects ontop of each frame using shaders.

Proposed Functionality

My initial thought on how this could function involves a few key steps:

1. Loading Video: A new function, perhaps load_video('filepath.mp4') (or whatever file format would be best suited), would be introduced to load a video file into the programs memory. This function would likely handle decoding and potentially caching of video data.

2. Getting Current Frame: A function like get_current_frame(time: f32) would allow the user to retrieve a specific frame from the loaded video at a given time. This function would return a Texture2D that can then be used with macroquad's existing drawing functions. The time parameter (in seconds) would allow the user to control which frame is extracted.

3. Drawing the Frame: Once a frame is retrieved as a texture, the user could use the standard draw_texture() family of functions to render it on the screen.

I have never done any programming using encoding and decoding video files before so I am not sure if this concept is the best approach or how exactly an implementation would function behind the scenes so I am open to discussing alternative approaches and understand that the maintainers will have a better understanding of the engine's architecture and potential implementation challenges, I thought it is probably worth putting in an request for now.

[KurlykovDanila]

Video decoding is usually done by specialized libraries(ffmpeg), they provide API for working with video formats. With their help, you can decode a separate frame, frame time, its number, resolution, frame format, etc. Macroquad provides Image and Texture2D structures for working with individual images. You can use third-party libraries for decoding and loading video, and rendering using macroquads.

A small example of how this could be done (quick example):

use ffmpeg_next::format::input;
use ffmpeg_next::software::scaling::{context::Context, flag::Flags};
use ffmpeg_next::util::frame::video::Video;
use macroquad::prelude::*;

#[macroquad::main("Video Player")]
async fn main() {
    ffmpeg_next::init().unwrap();
    let mut ictx = input(&"video.mp4").unwrap();
    let input = ictx
        .streams()
        .best(ffmpeg_next::media::Type::Video)
        .unwrap();
    let video_stream_index = input.index();

    let mut decoder = ffmpeg_next::codec::Context::from_parameters(input.parameters())
        .unwrap()
        .decoder()
        .video()
        .unwrap();

    let mut scaler = Context::get(
        decoder.format(),
        decoder.width(),
        decoder.height(),
        ffmpeg_next::format::Pixel::RGBA,
        decoder.width(),
        decoder.height(),
        Flags::BILINEAR,
    )
    .unwrap();

     let texture = Texture2D::from_image(&Image::gen_image_color(
        decoder.width() as u16,
        decoder.height() as u16,
        WHITE,
    ));
    texture.set_filter(FilterMode::Linear);

    loop {
        if let Some((stream, packet)) = ictx.packets().next() {
            if stream.index() == video_stream_index {
                decoder.send_packet(&packet).unwrap();
                let mut decoded = Video::empty();
                if decoder.receive_frame(&mut decoded).is_ok() {
                    let mut rgb_frame = Video::empty();
                    scaler.run(&decoded, &mut rgb_frame).unwrap();
       
                    texture.update(&Image {
                        bytes: rgb_frame.data(0).to_vec(),
                        width: rgb_frame.width() as u16,
                        height: rgb_frame.height() as u16,
                    });
                }
            }
        }

        clear_background(BLACK);
        draw_texture(&texture, 0.0, 0.0, WHITE);
        draw_fps();

        next_frame().await;
    }
}