Forkoz

Changed the fp32s near KV_cache to FP16 and didn't experience any loss in quality. Unfortunately the memory stayed about the same. A lot of other calculations are also done in...

Where else did you change it? I added both model.half and changed the float32s to float16s. The memory can still spike.

It worked for me but I'm not sure why it was locked to only the demo audio. I just made it editable and then the UI works.

``` with gr.Row(): with gr.Column(scale=2): input_audio = gr.Audio(value="./demo/84_121550_000074_000000.wav", label="Input Audio", type="filepath", interactive=False) with gr.Group(): ``` I just changed interactive to true.

It worked like this. I didn't check the console, sometimes there are exceptions. Usually reloading fixes it. I was using it all day yesterday after making it listen on more...

New version is great. The voices sound a lot better. Only thing is that I get a bit of the last word it's continuing from in my actual prompt and...

whisperX is missing some arguments so I haven't been able to try it. Using larger whisper models also helped with the timestamps. I guess the only other foible I noticed...

I got it all working, had to use the correct version of whisperx. The one from pip didn't work. Had to use git version. Sometimes I get a bug that...

BTW, I had already used the new model this morning. Output is fairly similar. I have not tried with less batches. Any ideas on why sometimes the output is really...

>The new model should work well with small batch sizes, and therefore requires less VRAM and inference time. It still gives better results with 4. Gave *ok* results at 1...