Feature: Silence skip

[some-git-user]

Pull Request Type

• [ ] Bugfix
• [x] Feature Implementation
• [ ] Documentation
• [ ] Other

Related issue

closes #1766

Description

This PR implements a simple silence skip logic, to dynamically determine silent video parts, related to the individual max volume of the audio source.

Screenshots

https://github.com/user-attachments/assets/f236ff1e-e788-4c45-83d1-bf425f2127d1

Testing

• Open any video (prefer a podcast with silent or mute parts between conversations).
• Toggle the function via the overflow menu Button.
• Watch/hear the player increasing playback speed to 3x between silent/mute audio parts and decreasing to 1x speed when audio is detected.
• Toggle the function again to deactivate and fallback to default 1x speed

Desktop

• OS: debian
• OS Version: 13
• FreeTube version: v0.23.11 Beta

Additional context

[efb4f5ff-1298-471a-8973-3d47447115dc]

Im a bit confused by this skip rate. Please elaborate why you chose for 2.5x instead of e.g. 10x

Lets say the ending of a certain video is 60sec it will take 24sec to skip the silence. Thats way too long imo.

Why not detect when there is no audio and detect when audio is playing again and instantly skip towards that timestamp instead of relying on playback rate to handle this

[some-git-user]

Im a bit confused by this skip rate. Please elaborate why you chose for 2.5x instead of e.g. 10x

Lets say the ending of a certain video is 60sec it will take 24sec to skip the silence. Thats way too long imo.

Why not detect when there is no audio and detect when audio is playing again and instantly skip towards that timestamp instead of relying on playback rate to handle this the

The main goal of this feature is to skip (accelerate) silence between conversations, or other silent parts, not to fast forward to the end of a video. I tried different speed steps with multiple videos and 2.5x was the best one in my tests, since faster steps could also "cut off" the beginning of the next non silent part.

I'm not sure how to "detect when there is no audio and detect when audio is playing again and instantly skip towards that timestamp" . Maybe I'm wrong here, but to know this you must buffer and analyze the whole video before the playback start, so the code knows the parts with no audio. I also tried to create a little UI, to let the user make individual changes for every video, to match the needed audio threshold, but I'm not familiar with Vue and the lack of Typescript made it even harder for me to get into this project.

[efb4f5ff-1298-471a-8973-3d47447115dc]

This implementation works really well, especially considering that our player is the limiting factor in this case.

[github-actions[bot]]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions[bot]]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[github-actions[bot]]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions[bot]]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[efb4f5ff-1298-471a-8973-3d47447115dc]

Hi @some-git-user could you rebase into dev branch so that the playback issues are resolved and this PR can be tested

[some-git-user]

@efb4f5ff-1298-471a-8973-3d47447115dc could you please point me to an code example to address following issue? "If i enable this feature within the player settings it turns is on everytime even if i turned it off in the watch session. What the user does within the Watch session should be retained. If user navigates to lets say the subscriptions page and clicks on a video that will be the start of a new Watch session and the setting should be enabled by default."

I can't find where to persist the watch session and ignore the Vuex.Store settings.

[efb4f5ff-1298-471a-8973-3d47447115dc]

It should remember its state in the watch session. Examples -> #5903 #6449

Hi @some-git-user i pointed you to a few examples some time ago. Did you look at these?

Also could you rebase again so that playback is functions properly

[efb4f5ff-1298-471a-8973-3d47447115dc]

Umm i think this isnt supposed to happen. It default to 10x now

https://github.com/user-attachments/assets/0c62341e-d865-4eae-b97a-a43112d8cb28

Discussion point: What should we do about people that are using a playback speed above 3x? I mean if i use 4x to watch a video and silent parts kicks in it probably shouldnt touch the playback speed right?

I think we probably should implement not touching the playback speed if its above 3x

[some-git-user]

@efb4f5ff-1298-471a-8973-3d47447115dc I used the user setting values for the fast and normal playback speed. I tested multiple values with various videos and it worked fine even with 10x forward speed. So in your case it's at 10x speed because this should also be your "Max Video Playback Rate"?

Do you have problems with speeds above 3x ? I can't unmute in your provided video examples. Are they recorded with sound?

[efb4f5ff-1298-471a-8973-3d47447115dc]

@some-git-user

Do you have problems with speeds above 3x ? I can't unmute in your provided video examples. Are they recorded with sound?

The arent recorded with sound but i think this case doesnt apply anymore because of the change you explained

I used the user setting values for the fast and normal playback speed. I tested multiple values with various videos and it worked fine even with 10x forward speed. So in your case it's at 10x speed because this should also be your "Max Video Playback Rate"?

Yes it is 10x. I will reevaluate!

[efb4f5ff-1298-471a-8973-3d47447115dc]

My Max video playback rate is set to 10x in the settings. Within the first 10sec of this video https://youtu.be/gT0rd69e5fo it will skip some silence but it also cuts of 2 words that have been said

[some-git-user]

@efb4f5ff-1298-471a-8973-3d47447115dc I think the 10x speed is too aggressive in some cases. The default 3x should be enough in most cases?

[efb4f5ff-1298-471a-8973-3d47447115dc]

i think 3x should be enough but i would like to hear from @PikachuEXE and @absidue before we decide on the right speed

[PikachuEXE]

I found it sound too rushy with it enabled, and I am not sure if it's better or worse to add N seconds before/after the speed up e.g. https://youtu.be/l8h4QgtNL0M And I am not sure if it also skip through something (but I only tested it briefly so could be wrong) On the speed I think 3x is fine (the gap/skip too much thing is bigger and once solved the speed while actual silence is not that important Need longer term testing

[efb4f5ff-1298-471a-8973-3d47447115dc]

3x seemed fine to me before it changed so maybe we should go back to that? Or are there other more logical speeds for this?

[some-git-user]

@efb4f5ff-1298-471a-8973-3d47447115dc would it be a problem,to let it be configurable?

[efb4f5ff-1298-471a-8973-3d47447115dc]

@some-git-user i prefer to not make this feature configurable. I prefer to go with good defaults instead of creating yet another slider/toggle in the settings

[github-actions[bot]]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions[bot]]

This PR is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 14 days.