yhq

Results 3 comments of yhq

Support Deepseek-V2

> HI @zwd003 This error occurred during the deployment process. How to solve it? Thanks! > > (RayWorkerWrapper pid=52311) ERROR 05-11 18:04:33 worker_base.py:145] File "/opt/vllm/vllm/model_executor/models/deepseek_v2.py", line 156, in forward (RayWorkerWrapper...

Support Deepseek-V2

> HI @zwd003 This error occurred during the deployment process. How to solve it? Thanks! > > (RayWorkerWrapper pid=52311) ERROR 05-11 18:04:33 worker_base.py:145] File "/opt/vllm/vllm/model_executor/models/deepseek_v2.py", line 156, in forward (RayWorkerWrapper...

H2O is slower than full

I run `python -m flexgen.flex_opt --gpu-batch-size 1 --overlap false --model facebook/opt-6.7b --path _DUMMY_ --prompt-len 512 --gen-len 512` and `python flex_opt.py --gpu-batch-size 1 --overlap false --hh-ratio 0.2 --hh-all --model facebook/opt-6.7b --path...