private-gpt errors when loading a document using two CUDAs

[jsgrover]

I installed a 2nd Cuda in the system. When I tried to load a document I get the following error

**.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/testuser/.pyenv/versions/chatPED/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/testuser/.pyenv/versions/chatPED/lib/python3.11/site-packages/transformers/models/bert/modeling_bert.py", line 1006, in forward embedding_output = self.embeddings( ^^^^^^^^^^^^^^^^ File "/home/testuser/.pyenv/versions/chatPED/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/testuser/.pyenv/versions/chatPED/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/testuser/.pyenv/versions/chatPED/lib/python3.11/site-packages/transformers/models/bert/modeling_bert.py", line 232, in forward inputs_embeds = self.word_embeddings(input_ids) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/testuser/.pyenv/versions/chatPED/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/testuser/.pyenv/versions/chatPED/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/testuser/.pyenv/versions/chatPED/lib/python3.11/site-packages/torch/nn/modules/sparse.py", line 162, in forward return F.embedding( ^^^^^^^^^^^^ File "/home/testuser/.pyenv/versions/chatPED/lib/python3.11/site-packages/torch/nn/functional.py", line 2233, in embedding return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)**

While LLM Chat works fine..

What am I doing wrong? I am a newbie to private-gpt and it's configuration. I haven't found anything about using 2 cuda, that needs to be configured.

[neofob]

@jsgrover : There is issue #1521 addressing this with a pending patch.