Modas Li

previous max seq len (32768), device is A100 40G --tensor-parallel-size 4 --gpu-memory-utilization 0.92 --max-model-len 512

not supported now

我之前用deepspeed example跑bloom也报错这个，听别人说是padding的问题

same issue， 意外截断

> 加了以后又碰到NotImplementedError: Cannot copy out of meta tensor; no data! 求教啊， @yangjianxin1 same issue

same problem when runnning with bloom， Add another argument max_seq_len=1600 ， does not work. please help me, thx

> Building wheels for collected packages: coati Building wheel for coati (setup.py) ... done Created wheel for coati: filename=coati-1.0.0-py3-none-any.whl size=63582 sha256=56804daa3dd2d332 f65cc00fed52545329f89de893f2b2370c5fdca85290c8ba Stored in directory: /tmp/pip-ephem-wheel-cache-ze69xz9j/wheels/e8/71/7c/45575a5c76c31c334b963c6a 660cb1b4f1881c79caeb57a45f Successfully built coati...

current env did not have coati, could you please advise me more,thx Attempting uninstall: pyyaml Found existing installation: PyYAML 3.13 ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project...

Thank you for your email !您好，你的邮件我已收到，我会尽快给你回复。

> Did you run this script（bash prepare_llama_cpp.sh）? > > You can check the parameters in the script prepare_llama_cpp.sh and set them to be exact. In the script, the python code...