useful-transformers icon indicating copy to clipboard operation
useful-transformers copied to clipboard

Efficient Inference of Transformer models

Results 27 useful-transformers issues
Sort by recently updated
recently updated
newest added

Hi, I was experimenting with this project and tried to use the whisper's "medium" model. It fails allocating memory on one of the `rknn_create_mem` after alllocating around 2GB of memory....

line 139 of whisper.py includes the assertion that `assert os.sched_getaffinity(os.getpid()) == set([4, 5, 6, 7])`. What is the motivation in limiting operations to select CPUs?

I downloaded the wav audio from here: https://voiceage.com/wbsamples/in_mono/Sports.wav And I used the following command: taskset -c 4-7 python -m useful_transformers.transcribe_wav Sports.wav The following is the result of the operation, which...

I'm getting a substantial amount of low confidence values on background noise and am looking to filter it out. How can I grab segment/translation temperatures or confidence?

Created a client ( audio recording buffer ) and server ( transcription backend ) example to test the possibility of doing async live transcription if it is fast enough. And...

Hi I have Whisper addon for home assistant, but i want to run ir on npu. It is posible with your program? Regards.

Trying to get this to work on my OrangePi 3b and running into `taskset: failed to set pid 2139's affinity: Invalid argument`

We run the model successfully on NPU, and we want to run it on GPU? How we should do ?

Enhance transcribe_wav.py with Output File Support and Default Behavior Preservation