whisper : update the "talk" example

[ggerganov]

The talk and talk.wasm examples have become a bit stale, using a very old implementation of gpt-2:

• https://github.com/ggerganov/whisper.cpp/tree/master/examples/talk
• https://github.com/ggerganov/whisper.cpp/tree/master/examples/talk.wasm

It would be nice to bring those examples up-to-date using the latest gpt-2 implementation from ggml: https://github.com/ggerganov/ggml/tree/master/examples/gpt-2

[Yuvraj-Lolage]

Hello,

I'm interested in updating the whisper.cpp repository to use the latest GPT-2 implementation from the ggml repository. Specifically, I plan to copy the contents of thewhisper.cpp of https://github.com/ggerganov/ggml/tree/master/examples/gpt-2 into the whisper.cpp file.

Before proceeding, I wanted to confirm if this is appropriate or not

[ggerganov]

No, you have to copy and adapt the gpt-2 implementation from the examples/gpt-2 in the ggml repo to the examples/talk in this repo. The whisper.cpp file does not need any changes

[tamo]

Something like this? I just copied ggml/examples/gpt-2/main-ctx.cpp to whisper.cpp/examples/talk/gpt-2.cpp https://github.com/tamo/whisper.cpp/compare/passfile...tamo:whisper.cpp:talkgpt2?expand=1 It builds but I'm not sure it works as expected. Feel free to use it if it helps.

[shivghai]

@ggerganov - tried a first pass at this, would appreciate any feedback (I'm a whisper/ggml noob). Works on an M1 Macbook Pro - https://github.com/ggerganov/whisper.cpp/pull/2384

[ggerganov]

@shivghai Thank you - will try to take a look soon

[xiaogz]

@shivghai I'm still encountering the same error message (https://github.com/ggerganov/whisper.cpp/issues/1272) when running the talk example in your branch compared to master at 5b1ce40 2024-09-17 server-use-OS-generated-temp-file-name-for-converted-files-2419 - Toliver. I'm using a mac studio m2 ultra on Sonoma 14.7.

Are you still working on your PR?

[shivghai]

@xiaogz the PR is up to date/ready - i just tried using my branch, and i can get it to run. Steps to run on my PR:

1. make talk
2. ./talk

How are you currently building/running?

[xiaogz]

@shivghai My bad! I made a git mistake. I fetched your forked branch properly and it works.

I also tried the fix myself too which differs from yours:

• copy over ggml's include and src at commit e7b2390 2024-09-20
• fix build breakage in whisper.cpp:

diff --git a/src/whisper.cpp b/src/whisper.cpp
index 35874aa..cf7c2c5 100644
--- a/src/whisper.cpp
+++ b/src/whisper.cpp
@@ -177,7 +177,7 @@ static bool ggml_graph_compute_helper(
                          int   n_threads,
          ggml_abort_callback   abort_callback,
                         void * abort_callback_data) {
-    struct ggml_cplan plan = ggml_graph_plan(graph, n_threads);
+    struct ggml_cplan plan = ggml_graph_plan(graph, n_threads, nullptr);

     plan.abort_callback      = abort_callback;
     plan.abort_callback_data = abort_callback_data;

• fix the buffer size calculations in examples/talk/gpt-2.cpp