render icon indicating copy to clipboard operation
render copied to clipboard

RecordMotion: native platform recorder

Open polarby opened this issue 2 years ago • 1 comments

With .recordMotion() users could take advantage of capturing widgets natively (including native views) and it could dramatically reduce capturing time from RepaintBoundary. This issue has been created to discuss the feasibility to record flutter programs natively in each platform. Adaption of this approach for capture motion for widgets that are present in the widget tree is also considerable.

The process of capturing could be archived by the following process:

  1. Capture full screen natively and write to a temporary file directory
  2. Capture the context of the render widget (size, position, transparency mask by capturing RepaintBoundary )
  3. Using flutters FFmpeg to crop the to-be-captured widget out of the screen recording (and apply a transparency mask).
  4. Convert cropped video to the expected file format

The following platform support needs to be checked.

  • [ ] Swift: ReplayKit
  • [ ] Web: java activity (browser-specific)
  • [ ] Android: Media Projection API
  • [ ] macOS
  • [ ] windows

Identify and Capture native widgets

If native rendering does not seem to make too much sense, in case no significant render time is reduced. It is considerable to take this method to directly identify native views and capture those individually, rather than through flutter render boundaries.

Definitive limitations:

  • dynamic transparency positions

polarby avatar Jan 16 '23 16:01 polarby

if we capture and crop entire screen that means we are limited with resolution of video with repaint boundry we can output a higher resolution output with crisp details

Kaizodo avatar Jul 17 '24 13:07 Kaizodo