mpv icon indicating copy to clipboard operation
mpv copied to clipboard

Playback mode that automatically skips silent parts.

Open LichenSymbiont opened this issue 6 years ago • 28 comments

Like this ffmpeg-based script that processes a file to remove silent parts with certain inputs for how silent it should be when cutting and such: github.com/carykh/jumpcutter

Another playback mode that would be useful is to detect silence from one of 2 audio channels, and playing mono.

There's probably a lot of things you can do with a "playback mode" feature.

LichenSymbiont avatar Jul 10 '19 10:07 LichenSymbiont

It'd be pretty trivial to write a filter which does this for audio only. But if video is involved, it would be much more complex.

ghost avatar Oct 24 '19 09:10 ghost

The audio filter should be disabled if there is any video stream. Is it possible to make an audio filter know whether there is video playing?

afcady avatar Aug 01 '20 20:08 afcady

I would want to use this filter in a video file too, so there should be an option or something like that, if that is even possible.

Zocker1999NET avatar Sep 14 '20 11:09 Zocker1999NET

I would like to see this implemented as well.

Would it be possible to have mpv run silencedetect on the file and store the output (internally or in a temp directory) to skip the detected sections maybe?

cankaratepe23 avatar Nov 26 '20 12:11 cankaratepe23

I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it

zkvsky avatar Dec 18 '20 05:12 zkvsky

What kinds of videos have silence which should be skipped? At all the clips I've ever watched, usually there's never real silence, and even if there was - I'd still like to watch the video...

avih avatar Dec 18 '20 13:12 avih

I don't know how it works but it's not a total silence, and surprisingly it's useful with many that I watch, game streams, any educational content, longform talks, reviews, obviously not with music. I'd love to have it for podcasts though. It doesn't skip/cut the content, just speeds it up

zkvsky avatar Dec 18 '20 15:12 zkvsky

What kinds of videos have silence which should be skipped? At all the clips I've ever watched, usually there's never real silence, and even if there was - I'd still like to watch the video...

For me personally, it's recordings of online lectures. I'm sure this would be useful to many people as well, and it can also be used for similar events such as webinars etc.

cankaratepe23 avatar Dec 18 '20 17:12 cankaratepe23

Yes, of course. I'm using it for lectures too, together with a small speed up. It's a real godsend, wish I knew about it earlier. It's a huge timesaver Hopefully someone will be able to jank it out of NewPipe, together with scaletempo2 it would be a killer

zkvsky avatar Dec 18 '20 18:12 zkvsky

Can someone upload a sample video someplace, and list the first few timestamps+duration of silence which should be removed?

I wonder if ffmpeg's silencedetect can be used in a script in a similar way which cropdetect is used.

avih avatar Dec 18 '20 19:12 avih

silencededetect been used that way for audio only things as per reddit, (outdated) The NewPipe one is pretty aggressive and it skips between words, you can see it happening in video but it's unnoticeable in audio and it's hard to timestamp, I'll look for something with longer pauses

zkvsky avatar Dec 18 '20 21:12 zkvsky

For example, it turns this https://youtu.be/T7SWETadMn0 16 min video into a 9 min video and from listening just to the audio I wouldn't be able to tell, it just sounds like she's speaking without ever catching breath

AFAIK they use this from ExoPlayer

zkvsky avatar Dec 18 '20 21:12 zkvsky

Please don't refer to other players and what they do or don't do right. Just give a sample clip and describe the first few timestamps (seconds resolution would be enough) and rough duration where such silence should be removed.

avih avatar Dec 18 '20 22:12 avih

That wasn't my intention, since they're open source I've linked directly to the code bit so it might be of use to someone. As I said it's hard to do because the implementation that I care about removes silence between words, so even shorter than a second, turning 2 hour podcasts into an hour one, without becoming exhausting to listen

zkvsky avatar Dec 19 '20 11:12 zkvsky

Personally I want it for music. Because I would like to skip the 10 to 15 seconds of silence that for some reason initiates many recordings of classical music. I just want the music to start instantly when I start the player.

afcady avatar Jan 11 '21 16:01 afcady

@afcady that one is achievable with lavfi skipsilence filter, it's pretty much what it's meant for.

zkvsky avatar Jan 12 '21 09:01 zkvsky

I would love to see this feature. I think a lot of people would. I'd love to be able to do something like:

mpv -detectSilence video.mp4

which results in mpv searching the audio stream for silence (for some given threshold) and opening it. It then set's like a"breakpoints" where the given silent part begins and ends and automatically skips it. Now people say that's a bad solution because you might skip content that's important but most people use it for lectures and how many lectures did you watch where the lecturer showed something without saying anything about it? Rarely, if ever.

To accommodate that, one would need motion detection - but that is resource heavy, so for the sake of efficiency I'd just stick with the above approach.

To give it more user friendlieness, one could enable the auto skip only on the first run through. So you watch the video, you come to a part where the video is silent for let's say 2 seconds and mpv automatically skips it. You now see that you somehow got lost and skip back. Since you skipped back manually, mpv won't skip forward again but let's you watch the previously skipped part.

Would make the life of educational content consumer a lot easier.

pascal-mueller avatar Feb 03 '21 10:02 pascal-mueller

I really hope this feature will be implemented.

IMHO, the user should have two options: to skip silent segments whenever they occur or to skip them only at the beginning and the end of the file.

  • The first option is useful for speeding up the silent parts while listening/watching (e.g., when listening to a talk or a conversation with a lot of silent moments).
  • The second one is useful for handling files in which the content actually begin a few seconds after the beginning of the file and ends a few seconds before its end (e.g., recorded example sentences, where there is a short delay between hitting ⏺ and speaking, and then hitting ⏹).

rwmpelstilzchen avatar Apr 24 '21 11:04 rwmpelstilzchen

I would like something similar to this browser extension: https://github.com/vantezzen/skip-silence

MinmoTech avatar Jul 13 '21 13:07 MinmoTech

This script solves the issue for SOME, videos, some meaning ones with subs/captions as it fast-forwards when no text is visible. [text can be made invisible easily] In practice it should mean most of the lectures and YT videos, works surprisingly well.

Edit. just found that there's one for audio too

zkvsky avatar Sep 09 '21 22:09 zkvsky

I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it

RIGHT? I need this for MPV and as a chrome extension!

pironside44 avatar Nov 10 '21 07:11 pironside44

T

I've recently noticed that NewPipe has such option for videos, but it's used for fast forwarding [auto fast-forwards when silence is detected ] and I'm completely addicted to it

RIGHT? I need this for MPV and as a chrome extension!

There is a chrome/firefox extension already called skip-silence mentioned above, it works fairly well If you want for it to work with your local videos just use something like Jellyfin as media provider, it will work with it too.

zkvsky avatar Nov 11 '21 03:11 zkvsky

Hi, I made a mpv script that uses the analysis data from https://github.com/WyattBlue/auto-editor to perform skipping.

https://gist.github.com/6ed6308be78dd8a2f58c9a9109fe050b

See the comments on the top for usage. If you have any question, put it in the gist comments (and not this issue thread).

Enjoy!

ishitatsuyuki avatar Jan 31 '22 04:01 ishitatsuyuki

Hi, I made a mpv script that uses the analysis data from https://github.com/WyattBlue/auto-editor to perform skipping.

https://gist.github.com/6ed6308be78dd8a2f58c9a9109fe050b

See the comments on the top for usage. If you have any question, put it in the gist comments (and not this issue thread).

Enjoy!

Thank you so much for this!. It works well. Finally, I can get back to watching lectures on the desktop.

zkvsky avatar Jan 31 '22 08:01 zkvsky

speed-transition.lua script skips to the next subtitle rather than audio. No one has mentioned it here, it achieves quite a similar purpose if your video is subtitled. You can also use Ctrl+RIGHT/LEFT to skip to the next/previous subtitle, this keybind comes out of the box in mpv. Edit: Found another: sub-skip.lua

notevenaperson avatar Mar 09 '22 13:03 notevenaperson

speed-transition.lua script skips to the next subtitle rather than audio. No one has mentioned it here, it achieves quite a similar purpose if your video is subtitled. You can also use Ctrl+RIGHT/LEFT to skip to the next/previous subtitle, this keybind comes out of the box in mpv. Edit: Found another: sub-skip.lua

Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too. Auto-editor based script works best for now, just needs speedup instead of cutting for more comfortable viewing.

zkvsky avatar Mar 11 '22 03:03 zkvsky

Unfortunately most of my classes/lectures hold subs for pauses, rendering these useless for me. They kick in with a delay too.

You might just re-do the timings with an automatic retimer like alass.

I agree that speedup is preferable to cutting.

notevenaperson avatar Mar 11 '22 09:03 notevenaperson

There is similar feature in NewPipe and it is very useful while watching lecature. Screenshot_2022-11-07-18-31-58-807_org schabi newpipe debug channeltabs_1_1

PriyanshuPriyadarshi avatar Nov 08 '22 02:11 PriyanshuPriyadarshi

I have tweaked the script https://gist.github.com/bitingsock/e8a56446ad9c1ed92d872aeb38edf124 (speed-transition-Audio.lua) and it works better now. Mainly the speed up to 5x during silences was too much and sometimes the upcoming audio would cut out. Now there is a gradual transition to 3x and works better.

Topping1 avatar Nov 22 '22 14:11 Topping1

Have you found a satisfying way to use ffmpeg's silencedetect / silenceremove in mpv?

I'm interested in experimenting with skipping over any silence in audio lasting more than a second or two.

ghost avatar Jun 15 '23 08:06 ghost