CapSoftware
feat: implement captions editor integration
hey @onyedikachi-david - thanks so much for the submission.
it doesn't seem to load for me, I get:
thread 'main' panicked at /Users/macbookuser/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/cmake-0.1.54/src/lib.rs:1119:5:
failed to execute command: No such file or directory (os error 2)
is `cmake` not installed?
build script failed, must exit now
stack backtrace:
0: 0x1029b6c1c - std::backtrace_rs::backtrace::libunwind::trace::h43235ac629484ce8
at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/std/src/../../backtrace/src/backtrace/libunwind.rs:116:5
1: 0x1029b6c1c - std::backtrace_rs::backtrace::trace_unsynchronized::h3bc78969d500b9fc
at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x1029b6c1c - std::sys::backtrace::_print_fmt::h66db558f95871955
at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/std/src/sys/backtrace.rs:66:9
3: 0x1029b6c1c - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h6e25c13836847e70
Also, we need the captions to be baked into the video frames that come from the renderer. Would you be able to check?
@onyedikachi-david it was on startup of the desktop app
I am assuming this, not sure:
let status = Command::new("ffmpeg")
.args([
"-i", &output_path.to_string_lossy(),
"-i", &srt_path.to_string_lossy(),
"-c:v", "copy",
"-c:a", "copy",
"-c:s", "mov_text",
"-metadata:s:s:0", "language=eng",
"-movflags", "+faststart",
"-map", "0:v",
"-map", "0:a",
"-map", "1:0",
"-y",
&temp_output.to_string_lossy(),
])
.status()
.map_err(|e| {
tracing::error!("FFmpeg command failed: {}", e);
format!("Failed to run FFmpeg: {}", e)
})?;
working quite well here:
Give me some time to find out what is wrong
right now you are:
- in the editor, rendering captions on top of canvas with html
- in the export, adding a subtitle track to the mp4
we need the subtitles to be rendered in the GPU as part of the actual frame. Not a subtitle track, but a physical implementation in the frame. This would allow us to change the text size, text appearance etc in the editor.
You can use a crate like glyphon to help with this.
right now you are:
- in the editor, rendering captions on top of canvas with html
- in the export, adding a subtitle track to the mp4
we need the subtitles to be rendered in the GPU as part of the actual frame. Not a subtitle track, but a physical implementation in the frame. This would allow us to change the text size, text appearance etc in the editor.
You can use a crate like glyphon to help with this.
Done, please review. Was a tough one.
https://github.com/user-attachments/assets/4a1e17e8-d4e5-44bf-91ed-ac982f2b0eb8
@richiemcilroy Here is a demo
Apologies for not looking at this for a while. I'm going to merge so we can start playing with the model downloading and transcription. Got a few things I want to do before releasing this as a feature including redoing our rendering backend.
Apologies for not looking at this for a while. I'm going to merge so we can start playing with the model downloading and transcription. Got a few things I want to do before releasing this as a feature including redoing our rendering backend.
It's all good.
Would love to help, paid or not, but I see that most issues/ are self assigned, do you allow contributions to such issues @Brendonovich