transformers.js icon indicating copy to clipboard operation
transformers.js copied to clipboard

WhisperModel initial_prompt

Open balbatross opened this issue 5 months ago • 3 comments

Feature request

https://github.com/huggingface/transformers/pull/28556

The core transformers library is moving to allow an extra parameter for the WhipserModel to bring initial_prompt in.

As one of the fastest, most supported implementations of Whisper in the browser it would be awesome to have this included in Transformers.js too

Motivation

Adding initial_prompt allows for use cases where the transcribed text needs to match jargon specific to an installation

Your contribution

I've tried getting to a point where I could make a PR with this but my lack of experience with both libraries has made that quite challenging. From the looks of it if we can get prompt_ids from the processor and pass them off to the WhisperModel it shouldn't be too hard to copy the core transformers implementation

balbatross avatar Sep 08 '24 22:09 balbatross