whisper.cpp icon indicating copy to clipboard operation
whisper.cpp copied to clipboard

Bring talk example up to date using the latest gpt-2 impl from ggml

Open shivghai opened this issue 5 months ago • 0 comments

https://github.com/users/ggerganov/projects/7?pane=issue&itemId=51613670

My attempt at porting over to the latest gpt-2 implementation from ggml. I'm a ggml + whisper.cpp noob but wanted to get this to work and looks like it works now

A few notes:

  • Sets default params for n_ctx [=2048] and n_gpu_layers [=0] to defaults mentioned in https://github.com/ggerganov/ggml/blob/master/examples/common.h#L19
  • I chmod +xed examples/talk/speak to get this to work, though I'm not too sure that's good practice to check it in that way
  • Code style is definitely off - gpt-2.h declares a lot of internals right now, happy to clear that up pending feedback. Seems like a lot of it is needed given that we need to know the structure of gpt2_context since in talk.cpp we use allocr = ggml_gallocr_new(ggml_backend_get_default_buffer_type(ctx_gpt->model.backend)); and an incomplete forward declaration will not be enough
  • Does not update talk.wasm (yet)

Runs on M1 MBP

shivghai avatar Aug 25 '24 21:08 shivghai