faster-whisper
faster-whisper copied to clipboard
SYSTRAN
Windows process crashes when the GPU model is unloaded
Thanks for your work first. It's useful.
Howerver, there's still something wrong. It returns -1073740791 (0xC0000409) when dealing with a audio file in chinese. I have defined a function in which the variable 'result' is used to accept the 'segments' returned by the fast-whisper. It's normal in this function, but abnormal after being returned by the function.
The line 'print(result)' works.
But after the result is returned, python returns -1073740791 (0xC0000409) and terminates
When changing the model or the language, it went properly.
Confused.
The same error is reported in #64 and is related to the cuDNN installation. Can you check that?
Really? But I have already installed Cudnn according to the Nivida documentation, and if I use medium-ct2 instead of large-v2-ct2, switch languages, or process other files, this type of problem will not occur.
Possibly you are running out of memory for this specific file. Can you try using compute_type="int8_float16"?
It still didn't work. When I tried using compute_type = "float32", it failed and returned "cuda out of memory". Howerver, this time it returned nothing but -1073740791 (0xC0000409). Seems like that it's not the reason.
I convert the audio file's format from aac to mp3. But it still didn't work. Seems like that it's not the format that's wrong.
the video file with command "ffmpeg -i "video file" -vn -ar 16000 "aac file"" "ffmpeg -i "video file" -vn -c:a libmp3lame -ar 16000 "mp3 file"" i extract the audio from the video
I don't reproduce the error on Windows 11 with CUDA 11.8.0 and cuDNN 8.8.1.
Are you using the latest version of faster-whisper?
The first thing I do when this error occurred is to update the faster-whisper
Seems like that it's difficult to reproduce the error. Since this problem only occurs with this unique file under certain conditions (I have also processed various other files since then, and the result is normal operation). Perhaps this issue can be put on hold? I will reopen this issue if the same error occurs in other files. Thank you for your help in so many days.
By coincidence, I discovered the cause of the error. When the model is unloaded, the program will crash.
Previously, model was a local variable and would automatically unload the model when the function ended. And the program crashes.
Now, model is a global variable and would automatically unload the model when the program ended.
Though the program will crash when unloading the model, it will happen after all things are finished now.
Howerver, I have no idea why the program will crash when unloading the model.
No, it doesn't.
It will when processing certain files (five out of approximately ninety files). If this error occurs when it processes a file, it will occur no matter how many times it processes the file. My environment is rtx 3060 laptop, windows 10, cuda11.7, cudnn8.8.0, python3.10.10, model large-v2, language 'zh'. Attempts have been made to reinstall the python environment or change the python version to 3.9.16, but this type of issue still exists. The model have been redownloaded, but issue exits.
If the model is manually unloaded after processing the file, the program will crash
I found a way to run the transcription in a separate process so that even though it exits that child process, it doesn't exit your main script. Here is a working example:
from multiprocessing import Process, Manager
from faster_whisper import WhisperModel
def work_log(argsList, returnList):
model_size = "large-v2"
model = WhisperModel(model_size, device="cuda", compute_type="float16")
segments, info = model.transcribe(*argsList, beam_size=5)
returnList[0] = [list(segments), info]
# workaround to deal with a termination issue: https://github.com/guillaumekln/faster-whisper/issues/71
def runWhisperSeperateProc(*args):
with Manager() as manager:
returnList = manager.list([None])
p = Process(target=work_log, args=[args, returnList]) # add return target to end of args list
p.start()
p.join()
p.close()
return returnList[0]
if __name__ == '__main__':
segments, info = runWhisperSeperateProc("audio.mp3")
print(segments, info)
I found a way to run the transcription in a separate process so that even though it exits that child process, it doesn't exit your main script. Here is a working example:
from multiprocessing import Process, Manager from faster_whisper import WhisperModel def work_log(argsList, returnList): model_size = "large-v2" model = WhisperModel(model_size, device="cuda", compute_type="float16") segments, info = model.transcribe(*argsList, beam_size=5) returnList[0] = [list(segments), info] # workaround to deal with a termination issue: https://github.com/guillaumekln/faster-whisper/issues/71 def runWhisperSeperateProc(*args): with Manager() as manager: returnList = manager.list([None]) p = Process(target=work_log, args=[args, returnList]) # add return target to end of args list p.start() p.join() p.close() return returnList[0] if __name__ == '__main__': segments, info = runWhisperSeperateProc("audio.mp3") print(segments, info)
same issue and open a Process to run works for me
@ProjectEGU @yslion @DoodleBears Are you all using the library on Windows?
I can now reproduce the issue on Windows.
It is somehow related to the temperature fallback. Can you try setting temperature=0?
Glad to know that the reason has been detected. With this setting, the program runs nomally. However, maybe it will produce slightly worsen result? I don't know.
@ProjectEGU @yslion @DoodleBears Are you all using the library on Windows?
Yes, I am using the library on Windows 10, I will try temperature=0 this evening
I have a possible fix for this issue in https://github.com/OpenNMT/CTranslate2/pull/1201, but I can't test on my Windows machine today. Can you help testing?
- Go to the build page
- Download the artifact "python-wheels"
- Extract the archive
- Install the Windows wheel matching your Python version with
pip install --force-reinstall <wheel file>
I have a possible fix for this issue in OpenNMT/CTranslate2#1201, but I can't test on my Windows machine today. Can you help testing?
- Go to the build page
- Download the artifact "python-wheels"
- Extract the archive
- Install the Windows wheel matching your Python version with
pip install --force-reinstall <wheel file>
Yes, I will try it now, by the way I try temperature=0, it works (process did not exit)
I have a possible fix for this issue in OpenNMT/CTranslate2#1201, but I can't test on my Windows machine today. Can you help testing?
- Go to the build page
- Download the artifact "python-wheels"
- Extract the archive
- Install the Windows wheel matching your Python version with
pip install --force-reinstall <wheel file>
I install the wheel.
- try to run without
temperature=0—— same issue (process still exit) - try to run with
temperature=0—— works
when using temperature=0:
I met segmentation fault BEFORE installing the wheel sometimes when I call the function below many times, don't know why
Sorry for did not keep the log, I remember it mentioned: cuBLAS and CUDA ...... segments fault, once I reproduce it, I will share the error log
def transcribe_speeches(self):
log.init_logging(debug=True)
# NOTE: 读取音频文件
logger.info(f"开始语音转文字")
whisper = WhisperModel(WHISPER_MODEL, device="cuda", compute_type="float16")
speeches_num = len(self.speeches)
for index, speech in enumerate(self.speeches):
logger.debug(f"开始识别 {speech.audio_path}")
speech_text = ''
# NOTE: 识别音频文件
segments, _ = whisper.transcribe(
audio=speech.audio_path,
language='zh',
vad_filter=False,
temperature=0,
initial_prompt='以下是普通话的句子。'
)
segments = list(segments)
if len(segments) == 0:
logger.warning(f"识别结果为空: {speech.audio_path}")
else:
speech_text = '，'.join([segment.text for segment in segments])
logger.info(f"识别结果({index+1}/{speeches_num}): {speech_text}")
self.speeches[index].text = speech_text
logger.info(f"结束语音转文字: {self.speeches}")
# queue.put(self.speeches)
# FIXME: 卸载模型后会导致程序终止
del whisper
import os
import subprocess
from faster_whisper import WhisperModel
files = os.listdir('input')
for file in files:
file = file.rsplit('.', 1)[0]
print(file)
command = f'ffmpeg -i \"input\\{file}.mp4\" -vn -ar 16000 -c:a libmp3lame -y \"tmp\\{file}.mp3\"'
subprocess.run(command, capture_output=True, check=True)
model = WhisperModel('large-v2', device='cuda', compute_type='float16')
segment, info = model.transcribe(f'tmp\\{file}.mp3', language=None)
for i in segment:
text = i.text
del model
print('finished')
same error
I'm not sure if this is directly related, but I get Segmentation fault error from time to time when I start to analyze the transcription segments via for segment in segments: .... I can't really pin down the precise location but it must be somewhere in WhisperModel.generate_segments and it happens only when my program tries to handle some remaining chunks at the end of a stream that are basically background noise.
Since I set temperature=0 it hasn't happened again.
@fquirin Are you also running on Windows with a GPU? If not, I’m not sure your issue is related. You can open another issue if you can share the audio and options triggering the crash.