screenclip icon indicating copy to clipboard operation
screenclip copied to clipboard

[TODO] Implement audio capture

Open 0Ky opened this issue 2 years ago • 0 comments

  • Capture audio data from a selectable option of sources (e.g., system audio, microphone, or both) using WASAPI (Windows Audio Session API).
  • Encode audio using the AAC codec.
  • Take the encoded video and audio streams and multiplex them into a single .mp4 container.
  • Ensure that audio and video are synchronized by testing the capture using an AV sync test video
  • UI/UX Considerations, provide appropriate UI controls:
    • Allow users to select audio sources (system audio, microphone, or both).
    • Provide feedback mechanisms, like audio meters, to show real-time audio levels.
    • Give users the option to toggle mute the audio source.

Note: We currently use video-rs a high-level abstraction of ffmpeg-next to encode video, unfortunately there is no audio API implemented, you may need to leverage ffmpeg-next for audio encoding/muxing. You may want to utilize entirely on another crate to encode/mux both video/audio sources, which will require rewriting the existing encode_frames function.

0Ky avatar Oct 10 '23 01:10 0Ky