oudoudou

> 4090 在跑RLHF阶段的train_ppo.py时，显存不够. 请问是不是运行的时候需要加什么参数？ > > torch.cuda.OutOfMemoryError： CUDA 内存不足。已尝试分配 30.00 MiB（GPU 0;23.62 GiB 总容量;已分配 21.75 GiB;3.12 MiB 可用;PyTorch 总共保留 22.47 GiB） 如果保留内存>>分配的内存，请尝试设置max_split_size_mb以避免碎片。请参阅有关内存管理和PYTORCH_CUDA_ALLOC_CONF的文档 > > llama-7b运行命令： > > ``` > CUDA_VISIBLE_DEVICES=0...

Is there a solution to this problem, the above link seems to be invalid

> > 是否要将 gptj = GPT4All （“ggml-gpt4all-j-v1.3-groovy”） 更改为 gptj = GPT4All（“mpt-7b-chat”， model_type=“mpt”）？ > > 我自己没有使用过 Python 绑定，只是使用 GUI，但是是的，这看起来是正确的。当然，您必须单独下载该模型。 ok，I see some model names by list_models() this function

> 啊，实际上，在我的文件浏览器中查找时，文件名是：`ggml-mpt-7b-chat.bin` You can take a look at it based on the official example, with .bin removed from the code ... GPT4All("ggml-gpt4all-j-v1.3-groovy").list_models()

> 是的，现在它可以工作了 But the results seemed to be different from what I expected

The model I use is "ggml-gpt4all-j-v1.3-groovy", and under the premise of changing print(response.decode('utf-8')) to print(response.decode('utf-8', errors='ignore')), I ask a question about python The answer is Python people. Python "Hello World!"....

OK thank you Model "mpt-7b-chat" I also tried, the problem is the same, there is garbled characters

Yes, I also saw it when I was debugging, he is divided, so even if it is applicable in some cases, but if the content of the question is changed,...

For the time being, I want to understand all the models first, and then choose the right model to debug and take a look