Falcon-40B thread problem.

[phalexo]

history: [['You are World renown expert on quantum mechanics and the Bell inequality. Do you understand? ', '']]

Exception in thread Thread-10 (generate_and_signal_complete): Traceback (most recent call last): File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/threading.py", line 1009, in _bootstrap_inner self.run() File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/threading.py", line 946, in run self._target(*self._args, **self._kwargs) File "/tmp/ipykernel_6551/331460570.py", line 122, in generate_and_signal_complete File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/transformers/generation/utils.py", line 1568, in generate return self.sample( File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/transformers/generation/utils.py", line 2615, in sample outputs = self( File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/developer/.cache/huggingface/modules/transformers_modules/falcon-40b-instruct/modelling_RW.py", line 759, in forward transformer_outputs = self.transformer( File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/developer/.cache/huggingface/modules/transformers_modules/falcon-40b-instruct/modelling_RW.py", line 654, in forward outputs = block( File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/developer/.cache/huggingface/modules/transformers_modules/falcon-40b-instruct/modelling_RW.py", line 396, in forward attn_outputs = self.self_attention( File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/developer/.cache/huggingface/modules/transformers_modules/falcon-40b-instruct/modelling_RW.py", line 252, in forward fused_qkv = self.query_key_value(hidden_states) # [batch_size, seq_length, 3 x hidden_size] File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/developer/.cache/huggingface/modules/transformers_modules/falcon-40b-instruct/modelling_RW.py", line 32, in forward ret = input @ self.weight.T RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasCreate(handle) Traceback (most recent call last): File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/gradio/routes.py", line 422, in run_predict output = await app.get_blocks().process_api( File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api result = await self.call_function( File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/gradio/blocks.py", line 1067, in call_function prediction = await utils.async_iteration(iterator) File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/gradio/utils.py", line 336, in async_iteration return await iterator.anext() File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/gradio/utils.py", line 329, in anext return await anyio.to_thread.run_sync( File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, *args) File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/gradio/utils.py", line 312, in run_sync_iterator_async return next(iterator) File "/tmp/ipykernel_6551/331460570.py", line 147, in bot for new_text in streamer: File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/transformers/generation/streamers.py", line 223, in next value = self.text_queue.get(timeout=self.timeout) File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/queue.py", line 179, in get raise Empty _queue.Empty

​