whisper.cpp icon indicating copy to clipboard operation
whisper.cpp copied to clipboard

What are the hyphens that are sometimes generated?

Open rohitkrishna094 opened this issue 11 months ago • 2 comments

I tried to transcribe a random youtube short. Converted it to wav file using ffmpeg. Here is the wav.

These are the results after trying it on 3 different models.

base: timeTaken ~ 6 seconds

Expand for results [00:00:00.000 --> 00:00:00.840] I have a question. [00:00:00.840 --> 00:00:01.680] - What's your question? [00:00:01.680 --> 00:00:02.880] - When you guys want the Christmas issue. [00:00:02.880 --> 00:00:04.600] - I have a whole Christmas list planned so. [00:00:04.600 --> 00:00:06.240] - What's at the top of your Christmas list issue? [00:00:06.240 --> 00:00:07.520] - Definitely a weenie dog. [00:00:07.520 --> 00:00:08.360] (gasps) [00:00:08.360 --> 00:00:10.360] - I love weenie dogs or I want a she shed. [00:00:10.360 --> 00:00:11.360] - What's a she shed? [00:00:11.360 --> 00:00:12.180] - A house. [00:00:12.180 --> 00:00:14.600] - I want to live on my own. [00:00:14.600 --> 00:00:16.600] - Oh my gosh, Harper, you're just 14. [00:00:16.600 --> 00:00:18.200] - I'm just 14, so I need a house. [00:00:18.200 --> 00:00:19.560] - Are you gonna live in your parents' backyard? [00:00:19.560 --> 00:00:20.560] - I am. [00:00:20.560 --> 00:00:21.400] - For like a year. [00:00:21.400 --> 00:00:22.600] - Show them how many it comes to dinner.

medium.en: timeTaken ~ 51 seconds

Expand for results [00:00:00.000 --> 00:00:01.580] I have a question. What's your question? [00:00:01.580 --> 00:00:02.840] What do you guys want for Christmas? [00:00:02.840 --> 00:00:04.260] I have a whole Christmas list planned. [00:00:04.260 --> 00:00:06.220] So what's at the top of your Christmas list this year? [00:00:06.220 --> 00:00:07.880] It's definitely a weenie dog. [00:00:07.880 --> 00:00:10.220] I love weenie dogs. Or I want a she shed. [00:00:10.220 --> 00:00:13.100] What's a she shed? A house? [00:00:13.100 --> 00:00:14.520] I want to live on my own. [00:00:14.520 --> 00:00:16.560] Oh my gosh, Harper, you're just 14. [00:00:16.560 --> 00:00:18.140] I'm just 14, so I need a house. [00:00:18.140 --> 00:00:19.520] Are you going to live in your parents backyard? [00:00:19.520 --> 00:00:21.400] I am. For like a year. [00:00:21.400 --> 00:00:23.400] You should come home when it comes to dinner time.

large-v3: timeTaken ~104 seconds

Expand for results [00:00:00.100 --> 00:00:00.940] - I have a question. [00:00:00.940 --> 00:00:01.780] - What? [00:00:01.780 --> 00:00:02.620] - What's your question? [00:00:02.620 --> 00:00:03.460] - When do you guys want for Christmas this year? [00:00:03.460 --> 00:00:04.620] - I have a whole Christmas list planned, so. [00:00:04.620 --> 00:00:06.280] - What's at the top of your Christmas list this year? [00:00:06.280 --> 00:00:07.260] - Definitely a weenie dog. [00:00:07.260 --> 00:00:09.260] - I love weenie dogs! [00:00:09.260 --> 00:00:10.400] - Or I want a she shed. [00:00:10.400 --> 00:00:11.360] - What's a she shed? [00:00:11.360 --> 00:00:12.200] - A house. [00:00:12.200 --> 00:00:14.600] - I wanna live on my own. [00:00:14.600 --> 00:00:16.640] - Oh my gosh, Harper, you're just 14. [00:00:16.640 --> 00:00:18.200] - I'm just 14, so I need a house. [00:00:18.200 --> 00:00:19.600] - Are you gonna live in your parents' backyard? [00:00:19.600 --> 00:00:20.600] - I am. [00:00:20.600 --> 00:00:21.440] For like a year. [00:00:21.440 --> 00:00:23.460] - Show them home when it comes to dinnertime, don't worry.

Only medium.en model doesnt have extra hyphens, but base and large-v3 have hyphens. What do they mean? Also for some reason (gasps) is not recognized in medium and large-v3 models, only in base is it recognized.

rohitkrishna094 avatar Mar 17 '24 07:03 rohitkrishna094

Generally they refer to a different speaker, e.g. - Hi, how are you? - I'm good. It's not proper diarization, but it's useful for my purposes, you might be able to suppress that token.

ooobo avatar Mar 18 '24 19:03 ooobo

Do you know how to suppress that token?

rohitkrishna094 avatar May 14 '24 00:05 rohitkrishna094