Hannan Komari

Tracing mismatch during conversion of Whisper model to ONNX using torch.onnx.export

3

I'm trying to convert Whisper model to onnx, so when exporting encoder of Whisper model to onnx by using torch.onnx.export: ``` mel = torch.zeros((1, 80, 3000)) encoder = model.get_encoder().to('cpu') audio_features...

Flax Whisper uses a lot of GPU memory

8

I'm using Flax whisper-medium and now it's ~3x faster rather than the pytorch deployment. but now it is allocating ~10x more GPU memory. loading Pytorch model takes ~3GB, but loading...

deploy whisper by passing last transcribed sentences to decoder's past_key values

4

I'm working on using whisper model for real time live transcription. I have to deploy audio chucks on model to have a sense of real time transcription, i.e. every 1...

Add support of output_scores to flax models

14

# What does this PR do? Flax models does not support `output_scores` when generate() method is called, despite the PyTorch models that fully supports this feature. It is tried to...

Get output_hidden_state and output_scores from Flax whisper model

7

I need whisper's output_scores and output_hidden_states as the result of generate() method. On Pytorch model, I can easily get the output_scores and output_hidden_states by setting these parameters in generate() method...

Got error of ModuleNotFoundError: No module named 'lightning_fabric'

3

Hi all I get the missing module error when importing `pyannote.audio` ``` /lib/python3.8/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or...

bug in graph_utils.py of zh ITN and decimal tagger of ar TN

3

In the `./nemo_text_processing/inverse_text_normalization/zh/graph_utils.py` line 79, `load_labels()` method is called but it is not imported, So it raises error. It could be simply resolved by adding the following method in `./nemo_text_processing/inverse_text_normalization/zh/utils.py`:...

malloc error on initialization of inverse normalizer

1

In the initialization of inverse normalizer for English language, sometimes the code crashed with the following error at the initialization: ```malloc(): unaligned tcache chunk detected``` I traced the code and...

Bug in WhisperTokenizer batch_decode, when set `skip_special_tokens=True` for FlaxWhisper model output

5

### System Info - `transformers` version: 4.43.0 - Platform: Linux-5.15.0-1061-nvidia-x86_64-with-glibc2.35 - Python version: 3.10.12 - Huggingface_hub version: 0.24.6 - Safetensors version: 0.4.4 - Accelerate version: not installed - Accelerate config:...

Fix flax whisper tokenizer bug

1

# What does this PR do? Fixes Bug when using whisper tokenizer for flax whisper model, according to the issue #32936 ## Before submitting - [ ] This PR fixes...